2023-10-19T10:25:29.984 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-19T10:25:29.985 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} --archive /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271 --verbose --name dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi --owner scheduled_dparmar@teuthology -- /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/orig.config.yaml 2023-10-19T10:25:30.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T10:25:30.112 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271 branch: wip-63105 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} email: null first_in_suite: false job_id: '7432271' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-63105 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - but it is still running - slow request - MDS_CLIENT_LATE_RELEASE - t responding to mclientcaps - file system flag refuse_client_session is set - Degraded data redundancy - MDS_CLIENTS_LAGGY - Reduced data availability sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-63105 sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d owner: scheduled_dparmar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4476 sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-63105 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVcW0lHH7vdor8OZ0/odg/BpSOxSYa/C/m1DtWPz9HScDV548DEXi+MR4XATtCyy/F3z0te96Co8q/6rkp4Ffw= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFTd5nvqZj0RT84qdMQXVb3bcX1g42IPAWhVYI7TaFUtS+MsLuT/FYYddm4d+bjWgOEDcH+V/K6pd1cofOY7wE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_recovery teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-19_10:17:02 tube: smithi user: dparmar verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-19T10:25:30.112 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa; will attempt to use it 2023-10-19T10:25:30.113 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa/tasks 2023-10-19T10:25:30.113 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-19T10:25:30.115 INFO:teuthology.task.internal:Checking packages... 2023-10-19T10:25:30.138 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6f3e6b48891b9b50415c0822b6b870797fcda16d' 2023-10-19T10:25:30.138 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-19T10:25:30.138 INFO:teuthology.packaging:ref: None 2023-10-19T10:25:30.139 INFO:teuthology.packaging:tag: None 2023-10-19T10:25:30.139 INFO:teuthology.packaging:branch: wip-63105 2023-10-19T10:25:30.139 INFO:teuthology.packaging:sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-19T10:25:30.140 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-63105 2023-10-19T10:25:30.409 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5610.g6f3e6b48 2023-10-19T10:25:30.410 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-19T10:25:30.417 INFO:teuthology.task.internal:no buildpackages task found 2023-10-19T10:25:30.417 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-19T10:25:30.423 INFO:teuthology.task.internal:Saving configuration 2023-10-19T10:25:30.439 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-19T10:25:30.445 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-19T10:25:30.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 10:20:40.335029', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVcW0lHH7vdor8OZ0/odg/BpSOxSYa/C/m1DtWPz9HScDV548DEXi+MR4XATtCyy/F3z0te96Co8q/6rkp4Ffw='} 2023-10-19T10:25:30.492 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 10:20:40.336202', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFTd5nvqZj0RT84qdMQXVb3bcX1g42IPAWhVYI7TaFUtS+MsLuT/FYYddm4d+bjWgOEDcH+V/K6pd1cofOY7wE='} 2023-10-19T10:25:30.492 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-19T10:25:30.498 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-19T10:25:30.498 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-19T10:25:30.499 INFO:teuthology.run_tasks:Running task console_log... 2023-10-19T10:25:30.566 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9a23483b80>, signals=[15]) 2023-10-19T10:25:30.567 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-19T10:25:30.573 INFO:teuthology.task.internal:Opening connections... 2023-10-19T10:25:30.574 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:25:30.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:25:30.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:25:30.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:25:30.757 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-19T10:25:30.764 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2023-10-19T10:25:30.787 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-10-19T10:25:30.788 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-10-19T10:25:30.853 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2023-10-19T10:25:30.935 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2023-10-19T10:25:30.935 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2023-10-19T10:25:30.936 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2023-10-19T10:25:30.936 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2023-10-19T10:25:30.936 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2023-10-19T10:25:30.936 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-19T10:25:30.937 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-10-19T10:25:30.937 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2023-10-19T10:25:30.937 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-19T10:25:30.937 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2023-10-19T10:25:30.938 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-19T10:25:30.938 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-19T10:25:30.938 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-19T10:25:30.939 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-10-19T10:25:30.970 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-10-19T10:25:30.988 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-10-19T10:25:30.988 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-10-19T10:25:31.044 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-10-19T10:25:31.045 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2023-10-19T10:25:31.045 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-10-19T10:25:31.045 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-10-19T10:25:31.045 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2023-10-19T10:25:31.045 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2023-10-19T10:25:31.046 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-19T10:25:31.046 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-10-19T10:25:31.046 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2023-10-19T10:25:31.046 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-19T10:25:31.046 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-10-19T10:25:31.047 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-19T10:25:31.047 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-19T10:25:31.047 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-19T10:25:31.047 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-10-19T10:25:31.070 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-19T10:25:31.086 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-19T10:25:31.091 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-19T10:25:31.092 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2023-10-19T10:25:31.094 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2023-10-19T10:25:31.110 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-19T10:25:31.116 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-19T10:25:31.117 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2023-10-19T10:25:31.152 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2023-10-19T10:25:31.186 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-19T10:25:31.237 INFO:teuthology.run_tasks:Running task kernel... 2023-10-19T10:25:31.252 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-19T10:25:31.252 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-19T10:25:31.253 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-19T10:25:31.253 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-19T10:25:31.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:25:31.269 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2023-10-19T10:25:31.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:25:31.270 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2023-10-19T10:25:31.325 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-361.el9.x86_64 2023-10-19T10:25:31.325 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.14.0-361.el9.x86_64 2023-10-19T10:25:31.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2023-10-19T10:25:31.328 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-361.el9.x86_64 2023-10-19T10:25:31.329 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-361.el9.x86_64 2023-10-19T10:25:31.329 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2023-10-19T10:25:33.441 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-19T10:25:33.495 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-19T10:25:34.836 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2023-10-19T10:25:35.059 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2023-10-19T10:25:35.936 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 7.9 MB 00:00 2023-10-19T10:25:37.870 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-10-19T10:25:38.274 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 7.9 MB 00:03 2023-10-19T10:25:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 25 MB/s | 18 MB 00:00 2023-10-19T10:25:42.229 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 5.7 MB 00:00 2023-10-19T10:25:43.730 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 14 kB 00:00 2023-10-19T10:25:44.961 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 5.7 MB 00:01 2023-10-19T10:25:45.248 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 14 MB/s | 19 MB 00:01 2023-10-19T10:25:47.209 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 14 kB 00:01 2023-10-19T10:25:48.477 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 16 MB/s | 19 MB 00:01 2023-10-19T10:25:49.906 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2023-10-19T10:25:52.088 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-19T10:25:52.088 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-19T10:25:52.184 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2023-10-19T10:25:52.191 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2023-10-19T10:25:52.192 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2023-10-19T10:25:52.192 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2023-10-19T10:25:52.192 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2023-10-19T10:25:52.192 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-376.el9 CentOS-BaseOS 5.3 M 2023-10-19T10:25:52.192 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2023-10-19T10:25:52.193 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-376.el9 CentOS-BaseOS 20 M 2023-10-19T10:25:52.193 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-376.el9 CentOS-BaseOS 38 M 2023-10-19T10:25:52.193 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-376.el9 CentOS-BaseOS 32 M 2023-10-19T10:25:52.193 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:25:52.193 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2023-10-19T10:25:52.194 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2023-10-19T10:25:52.194 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2023-10-19T10:25:52.194 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:25:52.195 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 95 M 2023-10-19T10:25:52.195 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 120 M 2023-10-19T10:25:52.195 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2023-10-19T10:25:52.516 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-376.el9.x86_64.rpm 16 MB/s | 5.3 MB 00:00 2023-10-19T10:25:52.877 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-19T10:25:53.100 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-376.el9.x86_64.rpm 22 MB/s | 20 MB 00:00 2023-10-19T10:25:54.050 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-core-5.14.0-376.el9.x86_6 21 MB/s | 32 MB 00:01 2023-10-19T10:25:54.633 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-5.14.0-376.el9.x86_64.rpm 15 MB/s | 38 MB 00:02 2023-10-19T10:25:54.636 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2023-10-19T10:25:54.637 INFO:teuthology.orchestra.run.smithi050.stdout:Total 39 MB/s | 95 MB 00:02 2023-10-19T10:25:54.637 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2023-10-19T10:25:54.950 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-19T10:25:54.951 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-19T10:25:55.045 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-10-19T10:25:55.048 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2023-10-19T10:25:55.048 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2023-10-19T10:25:55.055 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2023-10-19T10:25:55.055 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2023-10-19T10:25:55.055 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2023-10-19T10:25:55.056 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2023-10-19T10:25:55.056 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-376.el9 CentOS-BaseOS 5.3 M 2023-10-19T10:25:55.056 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2023-10-19T10:25:55.056 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-core x86_64 5.14.0-376.el9 CentOS-BaseOS 20 M 2023-10-19T10:25:55.056 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules x86_64 5.14.0-376.el9 CentOS-BaseOS 38 M 2023-10-19T10:25:55.057 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-core x86_64 5.14.0-376.el9 CentOS-BaseOS 32 M 2023-10-19T10:25:55.057 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:25:55.057 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2023-10-19T10:25:55.057 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2023-10-19T10:25:55.057 INFO:teuthology.orchestra.run.smithi183.stdout:Install 4 Packages 2023-10-19T10:25:55.058 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:25:55.058 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 95 M 2023-10-19T10:25:55.058 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 120 M 2023-10-19T10:25:55.059 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2023-10-19T10:25:56.109 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2023-10-19T10:25:56.110 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2023-10-19T10:25:56.370 INFO:teuthology.orchestra.run.smithi183.stdout:(1/4): kernel-5.14.0-376.el9.x86_64.rpm 4.0 MB/s | 5.3 MB 00:01 2023-10-19T10:25:57.962 INFO:teuthology.orchestra.run.smithi183.stdout:(2/4): kernel-core-5.14.0-376.el9.x86_64.rpm 6.9 MB/s | 20 MB 00:02 2023-10-19T10:25:58.590 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2023-10-19T10:25:58.603 INFO:teuthology.orchestra.run.smithi183.stdout:(3/4): kernel-modules-core-5.14.0-376.el9.x86_6 14 MB/s | 32 MB 00:02 2023-10-19T10:25:59.251 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-376.el9.x86_64 1/4 2023-10-19T10:25:59.320 INFO:teuthology.orchestra.run.smithi183.stdout:(4/4): kernel-modules-5.14.0-376.el9.x86_64.rpm 8.8 MB/s | 38 MB 00:04 2023-10-19T10:25:59.323 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2023-10-19T10:25:59.323 INFO:teuthology.orchestra.run.smithi183.stdout:Total 22 MB/s | 95 MB 00:04 2023-10-19T10:25:59.324 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2023-10-19T10:25:59.384 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-19T10:25:59.703 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2023-10-19T10:25:59.703 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2023-10-19T10:26:00.673 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2023-10-19T10:26:00.674 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2023-10-19T10:26:02.274 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-19T10:26:02.455 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-19T10:26:03.034 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2023-10-19T10:26:03.644 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-modules-core-5.14.0-376.el9.x86_64 1/4 2023-10-19T10:26:03.762 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-19T10:26:06.481 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-19T10:26:06.647 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-19T10:26:07.423 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-19T10:26:07.524 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:11.228 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-19T10:26:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:12.592 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:15.963 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:43.273 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:43.317 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:43.738 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:43.739 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-376.el9.x86_64 1/4 2023-10-19T10:26:43.739 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-19T10:26:43.739 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-19T10:26:44.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:44.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:26:44.192 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2023-10-19T10:26:44.192 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-376.el9.x86_64 2023-10-19T10:26:44.192 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-376.el9.x86_64 2023-10-19T10:26:44.193 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-376.el9.x86_64 2023-10-19T10:26:44.193 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-376.el9.x86_64 2023-10-19T10:26:44.193 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:26:44.193 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2023-10-19T10:26:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:44.603 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:44.693 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2023-10-19T10:26:44.834 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-19T10:26:44.834 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-19T10:26:44.835 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-376.el9.x86_64 2023-10-19T10:26:44.835 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-19T10:26:44.835 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-19T10:26:44.836 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-19T10:26:44.836 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2023-10-19T10:26:44.836 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2023-10-19T10:26:44.946 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-19T10:26:44.946 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-19T10:26:44.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2023-10-19T10:26:44.993 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2023-10-19T10:26:44.994 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2023-10-19T10:26:44.994 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2023-10-19T10:26:44.994 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2023-10-19T10:26:44.994 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2023-10-19T10:26:44.995 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-19T10:26:44.995 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2023-10-19T10:26:44.995 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2023-10-19T10:26:44.995 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2023-10-19T10:26:44.995 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-19T10:26:44.996 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-19T10:26:44.996 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-19T10:26:44.996 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-19T10:26:44.996 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2023-10-19T10:26:44.996 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2023-10-19T10:26:44.997 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-19T10:26:44.997 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2023-10-19T10:26:44.997 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2023-10-19T10:26:44.997 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:26:44.997 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-19T10:26:44.998 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-19T10:26:44.998 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-19T10:26:44.998 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2023-10-19T10:26:44.998 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:26:44.998 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2023-10-19T10:26:44.999 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-19T10:26:45.000 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-376.el9.x86_64 2023-10-19T10:26:45.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-19T10:26:45.050 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:45.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-376.el9.x86_64 1/4 2023-10-19T10:26:45.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-19T10:26:45.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-19T10:26:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-19T10:26:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:26:45.444 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2023-10-19T10:26:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-5.14.0-376.el9.x86_64 2023-10-19T10:26:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-core-5.14.0-376.el9.x86_64 2023-10-19T10:26:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-5.14.0-376.el9.x86_64 2023-10-19T10:26:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-core-5.14.0-376.el9.x86_64 2023-10-19T10:26:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:26:45.445 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-10-19T10:26:45.544 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2023-10-19T10:26:45.956 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2023-10-19T10:26:46.103 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-19T10:26:46.104 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-19T10:26:46.104 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-376.el9.x86_64 2023-10-19T10:26:46.104 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-19T10:26:46.105 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-19T10:26:46.105 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-19T10:26:46.105 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2023-10-19T10:26:46.105 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2023-10-19T10:26:46.209 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-19T10:26:46.210 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-19T10:26:46.211 DEBUG:teuthology.orchestra.run.smithi183:> sudo rpm -qi grub2-tools 2023-10-19T10:26:46.245 INFO:teuthology.orchestra.run.smithi183.stdout:Name : grub2-tools 2023-10-19T10:26:46.245 INFO:teuthology.orchestra.run.smithi183.stdout:Epoch : 1 2023-10-19T10:26:46.245 INFO:teuthology.orchestra.run.smithi183.stdout:Version : 2.06 2023-10-19T10:26:46.246 INFO:teuthology.orchestra.run.smithi183.stdout:Release : 68.el9 2023-10-19T10:26:46.246 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: x86_64 2023-10-19T10:26:46.246 INFO:teuthology.orchestra.run.smithi183.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-19T10:26:46.246 INFO:teuthology.orchestra.run.smithi183.stdout:Group : Unspecified 2023-10-19T10:26:46.247 INFO:teuthology.orchestra.run.smithi183.stdout:Size : 8264254 2023-10-19T10:26:46.247 INFO:teuthology.orchestra.run.smithi183.stdout:License : GPLv3+ 2023-10-19T10:26:46.247 INFO:teuthology.orchestra.run.smithi183.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-19T10:26:46.247 INFO:teuthology.orchestra.run.smithi183.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-19T10:26:46.247 INFO:teuthology.orchestra.run.smithi183.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-19T10:26:46.248 INFO:teuthology.orchestra.run.smithi183.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-19T10:26:46.248 INFO:teuthology.orchestra.run.smithi183.stdout:Packager : builder@centos.org 2023-10-19T10:26:46.248 INFO:teuthology.orchestra.run.smithi183.stdout:Vendor : CentOS 2023-10-19T10:26:46.248 INFO:teuthology.orchestra.run.smithi183.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-19T10:26:46.248 INFO:teuthology.orchestra.run.smithi183.stdout:Summary : Support tools for GRUB. 2023-10-19T10:26:46.249 INFO:teuthology.orchestra.run.smithi183.stdout:Description : 2023-10-19T10:26:46.249 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:26:46.249 INFO:teuthology.orchestra.run.smithi183.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-19T10:26:46.249 INFO:teuthology.orchestra.run.smithi183.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-19T10:26:46.249 INFO:teuthology.orchestra.run.smithi183.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-19T10:26:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:hardware devices. 2023-10-19T10:26:46.250 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:26:46.250 INFO:teuthology.orchestra.run.smithi183.stdout:This subpackage provides tools for support of all platforms. 2023-10-19T10:26:46.251 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-19T10:26:46.251 INFO:teuthology.task.kernel:Updating grub on smithi183 to boot 5.14.0-376.el9.x86_64 2023-10-19T10:26:46.251 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-19T10:26:46.844 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2023-10-19T10:26:46.898 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-19T10:26:46.945 INFO:teuthology.orchestra.run.smithi050.stderr:done 2023-10-19T10:26:46.947 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-10-19T10:26:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.KEOTPqKtlB 2023-10-19T10:26:46.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KEOTPqKtlB 2023-10-19T10:26:47.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.KEOTPqKtlB 2023-10-19T10:26:47.180 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.KEOTPqKtlB is 6KB 2023-10-19T10:26:47.228 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.KEOTPqKtlB 2023-10-19T10:26:47.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2023-10-19T10:26:47.311 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-19T10:26:47.312 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-19T10:26:47.312 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-19T10:26:47.312 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64.conf 2023-10-19T10:26:47.313 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64 2023-10-19T10:26:47.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2023-10-19T10:26:48.346 INFO:teuthology.orchestra.run.smithi183.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-19T10:26:48.379 INFO:teuthology.orchestra.run.smithi183.stderr:done 2023-10-19T10:26:48.381 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-10-19T10:26:48.396 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.GQkKKSjUZO 2023-10-19T10:26:48.397 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GQkKKSjUZO 2023-10-19T10:26:48.466 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.GQkKKSjUZO 2023-10-19T10:26:48.614 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.GQkKKSjUZO is 6KB 2023-10-19T10:26:48.633 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.GQkKKSjUZO 2023-10-19T10:26:48.647 DEBUG:teuthology.orchestra.run.smithi183:> sudo /bin/ls /boot/loader/entries || true 2023-10-19T10:26:48.711 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-19T10:26:48.711 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-19T10:26:48.711 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-19T10:26:48.712 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64.conf 2023-10-19T10:26:48.713 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64 2023-10-19T10:26:48.877 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2023-10-19T10:27:17.494 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-19T10:27:17.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-19T10:27:17.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:27:18.881 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-19T10:27:18.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-19T10:27:18.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:28:17.538 DEBUG:teuthology.orchestra.remote:timed out 2023-10-19T10:28:18.886 DEBUG:teuthology.orchestra.remote:timed out 2023-10-19T10:28:26.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-19T10:28:26.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:28:26.925 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-10-19T10:28:27.306 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-19T10:28:27.306 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-376.el9.x86_64"... 2023-10-19T10:28:27.306 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2023-10-19T10:28:27.321 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-376.el9.x86_64 2023-10-19T10:28:27.322 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-376.el9.x86_64 vs 5.14.0-376.el9.x86_64 2023-10-19T10:28:27.322 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-19T10:28:27.322 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-19T10:28:27.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-19T10:28:27.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:28:28.323 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-19T10:28:28.324 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-19T10:28:28.408 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2023-10-19T10:28:28.461 DEBUG:teuthology.parallel:result is None 2023-10-19T10:28:29.243 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-10-19T10:28:29.681 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-19T10:28:29.681 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-376.el9.x86_64"... 2023-10-19T10:28:29.681 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2023-10-19T10:28:29.695 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-376.el9.x86_64 2023-10-19T10:28:29.696 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-376.el9.x86_64 vs 5.14.0-376.el9.x86_64 2023-10-19T10:28:29.696 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-19T10:28:29.696 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-19T10:28:30.697 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-19T10:28:30.698 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-19T10:28:30.796 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2023-10-19T10:28:30.836 DEBUG:teuthology.parallel:result is None 2023-10-19T10:28:30.836 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-19T10:28:30.845 INFO:teuthology.task.internal:Creating test directory... 2023-10-19T10:28:30.846 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-19T10:28:30.849 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-19T10:28:30.863 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-19T10:28:30.870 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-19T10:28:30.884 INFO:teuthology.task.internal:Creating archive directory... 2023-10-19T10:28:30.885 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-19T10:28:30.903 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-19T10:28:31.012 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-19T10:28:31.019 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-19T10:28:31.020 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-19T10:28:31.022 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-19T10:28:31.055 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-19T10:28:31.068 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-19T10:28:31.076 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-19T10:28:31.094 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-19T10:28:31.095 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-19T10:28:31.103 INFO:teuthology.task.internal:Configuring sudo... 2023-10-19T10:28:31.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-19T10:28:31.106 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-19T10:28:31.136 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-19T10:28:31.146 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-19T10:28:31.146 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-19T10:28:31.175 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-19T10:28:31.191 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-19T10:28:31.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-19T10:28:31.327 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-19T10:28:31.369 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-19T10:28:31.410 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:28:31.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-19T10:28:31.477 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-19T10:28:31.519 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-19T10:28:31.570 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-19T10:28:31.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-19T10:28:31.693 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:28:31.693 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-19T10:28:31.761 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2023-10-19T10:28:31.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2023-10-19T10:28:31.814 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-19T10:28:31.862 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-19T10:28:32.290 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-19T10:28:32.299 INFO:teuthology.task.internal:Starting timer... 2023-10-19T10:28:32.299 INFO:teuthology.run_tasks:Running task pcp... 2023-10-19T10:28:32.309 INFO:teuthology.run_tasks:Running task selinux... 2023-10-19T10:28:32.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2023-10-19T10:28:32.396 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2023-10-19T10:28:32.398 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2023-10-19T10:28:32.454 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2023-10-19T10:28:32.456 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-19T10:28:32.457 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2023-10-19T10:28:32.476 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2023-10-19T10:28:32.477 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2023-10-19T10:28:32.492 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2023-10-19T10:28:32.493 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2023-10-19T10:28:32.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-19T10:28:32.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:28:32.549 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-19T10:28:32.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:28:32.576 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-19T10:28:32.577 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2023-10-19T10:28:32.628 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2023-10-19T10:28:32.657 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-19T10:28:32.666 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-19T10:28:32.672 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-19T10:28:32.782 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-19T10:28:32.799 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-19T10:28:32.800 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-19T10:34:01.372 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2023-10-19T10:34:01.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-19T10:34:01.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:34:01.473 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-10-19T10:34:01.544 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-10-19T10:34:01.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-19T10:34:01.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T10:34:01.628 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-10-19T10:34:01.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-19T10:34:01.698 INFO:teuthology.run_tasks:Running task clock... 2023-10-19T10:34:01.711 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-19T10:34:01.711 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-19T10:34:01.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-19T10:34:01.714 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-19T10:34:01.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-19T10:34:01.747 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-19T10:34:01.767 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-19T10:34:01.769 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-19T10:34:01.790 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-19T10:34:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2023-10-19T10:34:01.816 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2023-10-19T10:34:01.821 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2023-10-19T10:34:01.830 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2023-10-19T10:34:01.841 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-19T10:34:01.846 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-19T10:34:01.861 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-19T10:34:01.862 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-19T10:34:01.905 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2023-10-19T10:34:01.908 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-19T10:34:01.908 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2023-10-19T10:34:01.909 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.909 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.909 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.909 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2023-10-19T10:34:01.913 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-19T10:34:01.913 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2023-10-19T10:34:01.913 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.913 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.913 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.914 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:34:01.914 INFO:teuthology.run_tasks:Running task install... 2023-10-19T10:34:01.922 DEBUG:teuthology.task.install:project ceph 2023-10-19T10:34:01.923 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}} 2023-10-19T10:34:01.923 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'} 2023-10-19T10:34:01.924 INFO:teuthology.task.install:Using flavor: default 2023-10-19T10:34:01.930 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-19T10:34:01.930 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-19T10:34:01.931 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d', 'tag': None, 'wait_for_package': False} 2023-10-19T10:34:01.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-19T10:34:01.933 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d', 'tag': None, 'wait_for_package': False} 2023-10-19T10:34:01.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-19T10:34:02.079 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/ 2023-10-19T10:34:02.080 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5610.g6f3e6b48 2023-10-19T10:34:02.084 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/ 2023-10-19T10:34:02.085 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5610.g6f3e6b48 2023-10-19T10:34:02.193 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-19T10:34:02.194 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:34:02.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-19T10:34:02.197 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-19T10:34:02.198 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:34:02.198 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-19T10:34:02.226 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-19T10:34:02.226 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6f3e6b48891b9b50415c0822b6b870797fcda16d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-19T10:34:02.229 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-19T10:34:02.229 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6f3e6b48891b9b50415c0822b6b870797fcda16d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-19T10:34:02.294 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-19T10:34:02.300 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-19T10:34:02.374 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-19T10:34:02.386 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-19T10:34:02.416 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2023-10-19T10:34:02.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2023-10-19T10:34:02.440 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2023-10-19T10:34:02.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2023-10-19T10:34:02.673 INFO:teuthology.orchestra.run.smithi050.stdout:72 files removed 2023-10-19T10:34:02.690 INFO:teuthology.orchestra.run.smithi183.stdout:72 files removed 2023-10-19T10:34:02.701 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-19T10:34:02.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-19T10:34:03.399 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 215 kB/s | 78 kB 00:00 2023-10-19T10:34:03.516 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 163 kB/s | 78 kB 00:00 2023-10-19T10:34:03.850 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-10-19T10:34:03.959 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-10-19T10:34:04.200 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2023-10-19T10:34:04.276 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2023-10-19T10:34:04.919 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - AppStream 9.2 MB/s | 6.3 MB 00:00 2023-10-19T10:34:04.978 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 9.5 MB/s | 6.3 MB 00:00 2023-10-19T10:34:06.504 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2023-10-19T10:34:06.528 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2023-10-19T10:34:07.836 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 7.9 MB 00:00 2023-10-19T10:34:08.797 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 7.9 MB 00:01 2023-10-19T10:34:11.005 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 18 MB 00:02 2023-10-19T10:34:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-10-19T10:34:15.737 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 5.7 MB 00:01 2023-10-19T10:34:16.338 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 5.7 MB 00:01 2023-10-19T10:34:17.138 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 14 kB 00:00 2023-10-19T10:34:17.714 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 14 kB 00:00 2023-10-19T10:34:19.624 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 19 MB 00:02 2023-10-19T10:34:24.114 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-19T10:34:26.403 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-19T10:34:26.403 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-19T10:34:26.414 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-19T10:34:26.414 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-19T10:34:26.415 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-19T10:34:26.416 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-19T10:34:26.418 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-19T10:34:26.420 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-19T10:34:26.421 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-19T10:34:26.489 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-10-19T10:34:26.517 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2023-10-19T10:34:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2023-10-19T10:34:26.517 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2023-10-19T10:34:26.518 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2023-10-19T10:34:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-19T10:34:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 6.5 k 2023-10-19T10:34:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 5.2 M 2023-10-19T10:34:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 849 k 2023-10-19T10:34:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 141 k 2023-10-19T10:34:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 1.5 M 2023-10-19T10:34:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 132 k 2023-10-19T10:34:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 1.7 M 2023-10-19T10:34:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 7.4 M 2023-10-19T10:34:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 47 k 2023-10-19T10:34:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 8.2 M 2023-10-19T10:34:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 40 M 2023-10-19T10:34:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 261 k 2023-10-19T10:34:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 225 k 2023-10-19T10:34:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 218 k 2023-10-19T10:34:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 18 k 2023-10-19T10:34:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-19T10:34:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-19T10:34:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-19T10:34:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 31 k 2023-10-19T10:34:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 666 k 2023-10-19T10:34:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 127 k 2023-10-19T10:34:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-19T10:34:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-19T10:34:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-19T10:34:26.523 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-19T10:34:26.523 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-19T10:34:26.523 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-19T10:34:26.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 162 k 2023-10-19T10:34:26.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 319 k 2023-10-19T10:34:26.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 298 k 2023-10-19T10:34:26.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 99 k 2023-10-19T10:34:26.525 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 85 k 2023-10-19T10:34:26.525 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 3.0 M 2023-10-19T10:34:26.525 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 169 k 2023-10-19T10:34:26.525 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-19T10:34:26.525 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2023-10-19T10:34:26.525 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-19T10:34:26.526 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-19T10:34:26.526 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-19T10:34:26.526 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-19T10:34:26.526 INFO:teuthology.orchestra.run.smithi183.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-19T10:34:26.527 INFO:teuthology.orchestra.run.smithi183.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-19T10:34:26.527 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-19T10:34:26.527 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 3.3 M 2023-10-19T10:34:26.527 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 3.0 M 2023-10-19T10:34:26.527 INFO:teuthology.orchestra.run.smithi183.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-19T10:34:26.527 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2023-10-19T10:34:26.528 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-19T10:34:26.528 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-19T10:34:26.528 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 19 M 2023-10-19T10:34:26.528 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 23 k 2023-10-19T10:34:26.528 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 2.1 M 2023-10-19T10:34:26.529 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 242 k 2023-10-19T10:34:26.529 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 4.4 M 2023-10-19T10:34:26.529 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 18 M 2023-10-19T10:34:26.529 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 15 k 2023-10-19T10:34:26.529 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 24 k 2023-10-19T10:34:26.530 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-19T10:34:26.530 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-19T10:34:26.530 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-19T10:34:26.530 INFO:teuthology.orchestra.run.smithi183.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-19T10:34:26.530 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-19T10:34:26.531 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-19T10:34:26.531 INFO:teuthology.orchestra.run.smithi183.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-19T10:34:26.531 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-19T10:34:26.531 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-19T10:34:26.531 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 163 k 2023-10-19T10:34:26.532 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-19T10:34:26.532 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-19T10:34:26.532 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-19T10:34:26.532 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-19T10:34:26.532 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-19T10:34:26.532 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 472 k 2023-10-19T10:34:26.533 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-19T10:34:26.533 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 4.7 M 2023-10-19T10:34:26.533 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-19T10:34:26.533 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-19T10:34:26.533 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-19T10:34:26.534 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-19T10:34:26.534 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-19T10:34:26.534 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-19T10:34:26.534 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-19T10:34:26.535 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-19T10:34:26.535 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-19T10:34:26.535 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-19T10:34:26.535 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-19T10:34:26.535 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-19T10:34:26.536 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 45 k 2023-10-19T10:34:26.536 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 127 k 2023-10-19T10:34:26.536 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-19T10:34:26.536 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-19T10:34:26.536 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-19T10:34:26.536 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-19T10:34:26.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-19T10:34:26.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-19T10:34:26.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-19T10:34:26.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-19T10:34:26.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-19T10:34:26.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-19T10:34:26.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-19T10:34:26.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-19T10:34:26.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-19T10:34:26.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-19T10:34:26.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-19T10:34:26.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-19T10:34:26.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-19T10:34:26.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-19T10:34:26.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-19T10:34:26.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-19T10:34:26.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-19T10:34:26.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-19T10:34:26.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-19T10:34:26.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-19T10:34:26.541 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-19T10:34:26.541 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-19T10:34:26.541 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-19T10:34:26.541 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-19T10:34:26.541 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-19T10:34:26.541 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-19T10:34:26.542 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-19T10:34:26.542 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-19T10:34:26.542 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-19T10:34:26.542 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-19T10:34:26.542 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-19T10:34:26.543 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-19T10:34:26.543 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-19T10:34:26.543 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-19T10:34:26.543 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-19T10:34:26.543 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-19T10:34:26.544 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-19T10:34:26.544 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-19T10:34:26.544 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-19T10:34:26.544 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-19T10:34:26.544 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-19T10:34:26.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-19T10:34:26.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-19T10:34:26.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-19T10:34:26.545 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-19T10:34:26.545 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-19T10:34:26.545 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-19T10:34:26.546 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-19T10:34:26.546 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-19T10:34:26.546 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-19T10:34:26.546 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2023-10-19T10:34:26.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-19T10:34:26.547 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:34:26.547 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2023-10-19T10:34:26.547 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2023-10-19T10:34:26.547 INFO:teuthology.orchestra.run.smithi183.stdout:Install 133 Packages 2023-10-19T10:34:26.547 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 10 Packages 2023-10-19T10:34:26.548 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:34:26.548 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 198 M 2023-10-19T10:34:26.549 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2023-10-19T10:34:27.490 INFO:teuthology.orchestra.run.smithi183.stdout:(1/143): ceph-18.0.0-5610.g6f3e6b48.el9.x86_64. 31 kB/s | 6.5 kB 00:00 2023-10-19T10:34:28.057 INFO:teuthology.orchestra.run.smithi183.stdout:(2/143): ceph-fuse-18.0.0-5610.g6f3e6b48.el9.x8 1.5 MB/s | 849 kB 00:00 2023-10-19T10:34:28.157 INFO:teuthology.orchestra.run.smithi183.stdout:(3/143): ceph-immutable-object-cache-18.0.0-561 1.4 MB/s | 141 kB 00:00 2023-10-19T10:34:28.415 INFO:teuthology.orchestra.run.smithi183.stdout:(4/143): ceph-common-18.0.0-5610.g6f3e6b48.el9. 17 MB/s | 19 MB 00:01 2023-10-19T10:34:28.524 INFO:teuthology.orchestra.run.smithi183.stdout:(5/143): ceph-mgr-18.0.0-5610.g6f3e6b48.el9.x86 13 MB/s | 1.5 MB 00:00 2023-10-19T10:34:28.783 INFO:teuthology.orchestra.run.smithi183.stdout:(6/143): ceph-mon-18.0.0-5610.g6f3e6b48.el9.x86 17 MB/s | 4.4 MB 00:00 2023-10-19T10:34:28.991 INFO:teuthology.orchestra.run.smithi183.stdout:(7/143): ceph-mds-18.0.0-5610.g6f3e6b48.el9.x86 2.5 MB/s | 2.1 MB 00:00 2023-10-19T10:34:29.260 INFO:teuthology.orchestra.run.smithi183.stdout:(8/143): ceph-base-18.0.0-5610.g6f3e6b48.el9.x8 2.6 MB/s | 5.2 MB 00:01 2023-10-19T10:34:29.358 INFO:teuthology.orchestra.run.smithi183.stdout:(9/143): ceph-selinux-18.0.0-5610.g6f3e6b48.el9 245 kB/s | 24 kB 00:00 2023-10-19T10:34:29.750 INFO:teuthology.orchestra.run.smithi183.stdout:(10/143): ceph-osd-18.0.0-5610.g6f3e6b48.el9.x8 18 MB/s | 18 MB 00:00 2023-10-19T10:34:29.800 INFO:teuthology.orchestra.run.smithi183.stdout:(11/143): cephfs-mirror-18.0.0-5610.g6f3e6b48.e 4.3 MB/s | 218 kB 00:00 2023-10-19T10:34:29.859 INFO:teuthology.orchestra.run.smithi183.stdout:(12/143): libcephfs-devel-18.0.0-5610.g6f3e6b48 536 kB/s | 31 kB 00:00 2023-10-19T10:34:29.959 INFO:teuthology.orchestra.run.smithi183.stdout:(13/143): libcephfs2-18.0.0-5610.g6f3e6b48.el9. 6.5 MB/s | 666 kB 00:00 2023-10-19T10:34:30.009 INFO:teuthology.orchestra.run.smithi183.stdout:(14/143): libcephsqlite-18.0.0-5610.g6f3e6b48.e 3.2 MB/s | 163 kB 00:00 2023-10-19T10:34:30.060 INFO:teuthology.orchestra.run.smithi183.stdout:(15/143): librados-devel-18.0.0-5610.g6f3e6b48. 2.5 MB/s | 127 kB 00:00 2023-10-19T10:34:30.118 INFO:teuthology.orchestra.run.smithi183.stdout:(16/143): libradosstriper1-18.0.0-5610.g6f3e6b4 7.9 MB/s | 472 kB 00:00 2023-10-19T10:34:30.477 INFO:teuthology.orchestra.run.smithi183.stdout:(17/143): librgw2-18.0.0-5610.g6f3e6b48.el9.x86 13 MB/s | 4.7 MB 00:00 2023-10-19T10:34:30.519 INFO:teuthology.orchestra.run.smithi183.stdout:(18/143): python3-ceph-argparse-18.0.0-5610.g6f 1.1 MB/s | 45 kB 00:00 2023-10-19T10:34:30.627 INFO:teuthology.orchestra.run.smithi183.stdout:(19/143): ceph-radosgw-18.0.0-5610.g6f3e6b48.el 5.0 MB/s | 8.2 MB 00:01 2023-10-19T10:34:30.653 INFO:teuthology.orchestra.run.smithi183.stdout:(20/143): python3-ceph-common-18.0.0-5610.g6f3e 948 kB/s | 127 kB 00:00 2023-10-19T10:34:30.703 INFO:teuthology.orchestra.run.smithi183.stdout:(21/143): python3-rados-18.0.0-5610.g6f3e6b48.e 6.3 MB/s | 319 kB 00:00 2023-10-19T10:34:30.728 INFO:teuthology.orchestra.run.smithi183.stdout:(22/143): python3-cephfs-18.0.0-5610.g6f3e6b48. 1.6 MB/s | 162 kB 00:00 2023-10-19T10:34:30.770 INFO:teuthology.orchestra.run.smithi183.stdout:(23/143): python3-rbd-18.0.0-5610.g6f3e6b48.el9 4.4 MB/s | 298 kB 00:00 2023-10-19T10:34:30.820 INFO:teuthology.orchestra.run.smithi183.stdout:(24/143): python3-rgw-18.0.0-5610.g6f3e6b48.el9 1.1 MB/s | 99 kB 00:00 2023-10-19T10:34:30.854 INFO:teuthology.orchestra.run.smithi183.stdout:(25/143): rbd-fuse-18.0.0-5610.g6f3e6b48.el9.x8 1.0 MB/s | 85 kB 00:00 2023-10-19T10:34:30.921 INFO:teuthology.orchestra.run.smithi183.stdout:(26/143): rbd-nbd-18.0.0-5610.g6f3e6b48.el9.x86 2.5 MB/s | 169 kB 00:00 2023-10-19T10:34:30.971 INFO:teuthology.orchestra.run.smithi183.stdout:(27/143): ceph-grafana-dashboards-18.0.0-5610.g 464 kB/s | 23 kB 00:00 2023-10-19T10:34:31.022 INFO:teuthology.orchestra.run.smithi183.stdout:(28/143): ceph-mgr-cephadm-18.0.0-5610.g6f3e6b4 2.6 MB/s | 132 kB 00:00 2023-10-19T10:34:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:(29/143): ceph-mgr-dashboard-18.0.0-5610.g6f3e6 9.6 MB/s | 1.7 MB 00:00 2023-10-19T10:34:31.539 INFO:teuthology.orchestra.run.smithi183.stdout:(30/143): rbd-mirror-18.0.0-5610.g6f3e6b48.el9. 4.2 MB/s | 3.0 MB 00:00 2023-10-19T10:34:31.672 INFO:teuthology.orchestra.run.smithi183.stdout:(31/143): ceph-mgr-modules-core-18.0.0-5610.g6f 1.8 MB/s | 242 kB 00:00 2023-10-19T10:34:31.789 INFO:teuthology.orchestra.run.smithi183.stdout:(32/143): ceph-mgr-rook-18.0.0-5610.g6f3e6b48.e 407 kB/s | 47 kB 00:00 2023-10-19T10:34:31.889 INFO:teuthology.orchestra.run.smithi183.stdout:(33/143): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-10-19T10:34:31.906 INFO:teuthology.orchestra.run.smithi183.stdout:(34/143): ceph-prometheus-alerts-18.0.0-5610.g6 125 kB/s | 15 kB 00:00 2023-10-19T10:34:31.957 INFO:teuthology.orchestra.run.smithi183.stdout:(35/143): cephadm-18.0.0-5610.g6f3e6b48.el9.noa 4.4 MB/s | 225 kB 00:00 2023-10-19T10:34:31.982 INFO:teuthology.orchestra.run.smithi183.stdout:(36/143): ceph-volume-18.0.0-5610.g6f3e6b48.el9 2.8 MB/s | 261 kB 00:00 2023-10-19T10:34:32.007 INFO:teuthology.orchestra.run.smithi183.stdout:(37/143): cephfs-top-18.0.0-5610.g6f3e6b48.el9. 362 kB/s | 18 kB 00:00 2023-10-19T10:34:32.149 INFO:teuthology.orchestra.run.smithi183.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 746 kB/s | 106 kB 00:00 2023-10-19T10:34:32.191 INFO:teuthology.orchestra.run.smithi183.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2023-10-19T10:34:32.216 INFO:teuthology.orchestra.run.smithi183.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2023-10-19T10:34:32.242 INFO:teuthology.orchestra.run.smithi183.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2023-10-19T10:34:32.342 INFO:teuthology.orchestra.run.smithi183.stdout:(42/143): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 17 kB 00:00 2023-10-19T10:34:32.450 INFO:teuthology.orchestra.run.smithi183.stdout:(43/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2023-10-19T10:34:32.475 INFO:teuthology.orchestra.run.smithi183.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 76 kB/s | 10 kB 00:00 2023-10-19T10:34:32.501 INFO:teuthology.orchestra.run.smithi183.stdout:(45/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-10-19T10:34:32.543 INFO:teuthology.orchestra.run.smithi183.stdout:(46/143): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2023-10-19T10:34:32.593 INFO:teuthology.orchestra.run.smithi183.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2023-10-19T10:34:32.618 INFO:teuthology.orchestra.run.smithi183.stdout:(48/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 625 kB/s | 46 kB 00:00 2023-10-19T10:34:32.652 INFO:teuthology.orchestra.run.smithi183.stdout:(49/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2023-10-19T10:34:32.677 INFO:teuthology.orchestra.run.smithi183.stdout:(50/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2023-10-19T10:34:32.702 INFO:teuthology.orchestra.run.smithi183.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2023-10-19T10:34:33.053 INFO:teuthology.orchestra.run.smithi183.stdout:(52/143): ceph-test-18.0.0-5610.g6f3e6b48.el9.x 11 MB/s | 40 MB 00:03 2023-10-19T10:34:33.203 INFO:teuthology.orchestra.run.smithi183.stdout:(53/143): openblas-openmp-0.3.21-2.el9.x86_64.r 9.0 MB/s | 4.7 MB 00:00 2023-10-19T10:34:33.253 INFO:teuthology.orchestra.run.smithi183.stdout:(54/143): postgresql-13.11-1.el9.x86_64.rpm 2.8 MB/s | 1.5 MB 00:00 2023-10-19T10:34:33.320 INFO:teuthology.orchestra.run.smithi183.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 3.0 MB/s | 811 kB 00:00 2023-10-19T10:34:33.346 INFO:teuthology.orchestra.run.smithi183.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 950 kB/s | 134 kB 00:00 2023-10-19T10:34:33.721 INFO:teuthology.orchestra.run.smithi183.stdout:(57/143): python3-devel-3.9.17-2.el9.x86_64.rpm 547 kB/s | 205 kB 00:00 2023-10-19T10:34:33.838 INFO:teuthology.orchestra.run.smithi183.stdout:(58/143): postgresql-server-13.11-1.el9.x86_64. 9.7 MB/s | 5.7 MB 00:00 2023-10-19T10:34:34.013 INFO:teuthology.orchestra.run.smithi183.stdout:(59/143): python3-jinja2-2.11.3-4.el9.noarch.rp 785 kB/s | 228 kB 00:00 2023-10-19T10:34:34.146 INFO:teuthology.orchestra.run.smithi183.stdout:(60/143): python3-babel-2.9.1-2.el9.noarch.rpm 7.0 MB/s | 5.8 MB 00:00 2023-10-19T10:34:34.172 INFO:teuthology.orchestra.run.smithi183.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 73 kB/s | 24 kB 00:00 2023-10-19T10:34:34.197 INFO:teuthology.orchestra.run.smithi183.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 99 kB/s | 18 kB 00:00 2023-10-19T10:34:34.222 INFO:teuthology.orchestra.run.smithi183.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-10-19T10:34:34.248 INFO:teuthology.orchestra.run.smithi183.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 637 kB/s | 32 kB 00:00 2023-10-19T10:34:34.506 INFO:teuthology.orchestra.run.smithi183.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2023-10-19T10:34:34.531 INFO:teuthology.orchestra.run.smithi183.stdout:(66/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 705 kB/s | 217 kB 00:00 2023-10-19T10:34:34.556 INFO:teuthology.orchestra.run.smithi183.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 567 kB/s | 174 kB 00:00 2023-10-19T10:34:34.582 INFO:teuthology.orchestra.run.smithi183.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 553 kB/s | 41 kB 00:00 2023-10-19T10:34:34.607 INFO:teuthology.orchestra.run.smithi183.stdout:(69/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-10-19T10:34:34.632 INFO:teuthology.orchestra.run.smithi183.stdout:(70/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2023-10-19T10:34:34.649 INFO:teuthology.orchestra.run.smithi183.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 43 kB 00:00 2023-10-19T10:34:34.841 INFO:teuthology.orchestra.run.smithi183.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 203 kB/s | 39 kB 00:00 2023-10-19T10:34:34.866 INFO:teuthology.orchestra.run.smithi183.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2023-10-19T10:34:35.033 INFO:teuthology.orchestra.run.smithi183.stdout:(74/143): uuid-1.6.2-55.el9.x86_64.rpm 294 kB/s | 56 kB 00:00 2023-10-19T10:34:35.058 INFO:teuthology.orchestra.run.smithi183.stdout:(75/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 330 kB/s | 63 kB 00:00 2023-10-19T10:34:35.265 INFO:teuthology.orchestra.run.smithi183.stdout:(76/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2023-10-19T10:34:35.291 INFO:teuthology.orchestra.run.smithi183.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 308 kB/s | 79 kB 00:00 2023-10-19T10:34:35.316 INFO:teuthology.orchestra.run.smithi183.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2023-10-19T10:34:35.341 INFO:teuthology.orchestra.run.smithi183.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2023-10-19T10:34:35.367 INFO:teuthology.orchestra.run.smithi183.stdout:(80/143): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2023-10-19T10:34:35.392 INFO:teuthology.orchestra.run.smithi183.stdout:(81/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2023-10-19T10:34:35.409 INFO:teuthology.orchestra.run.smithi183.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 124 kB 00:00 2023-10-19T10:34:35.434 INFO:teuthology.orchestra.run.smithi183.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.7 MB/s | 113 kB 00:00 2023-10-19T10:34:35.459 INFO:teuthology.orchestra.run.smithi183.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 3.7 MB/s | 188 kB 00:00 2023-10-19T10:34:35.684 INFO:teuthology.orchestra.run.smithi183.stdout:(85/143): python3-cryptography-36.0.1-4.el9.x86 4.5 MB/s | 1.2 MB 00:00 2023-10-19T10:34:35.727 INFO:teuthology.orchestra.run.smithi183.stdout:(86/143): elfutils-libelf-devel-0.189-3.el9.x86 84 kB/s | 22 kB 00:00 2023-10-19T10:34:35.946 INFO:teuthology.orchestra.run.smithi183.stdout:(87/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 209 kB/s | 45 kB 00:00 2023-10-19T10:34:36.064 INFO:teuthology.orchestra.run.smithi183.stdout:(88/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 219 kB/s | 26 kB 00:00 2023-10-19T10:34:36.202 INFO:teuthology.orchestra.run.smithi183.stdout:(89/143): automake-1.16.2-8.el9.noarch.rpm 898 kB/s | 689 kB 00:00 2023-10-19T10:34:36.261 INFO:teuthology.orchestra.run.smithi183.stdout:(90/143): jq-1.6-15.el9.x86_64.rpm 324 kB/s | 187 kB 00:00 2023-10-19T10:34:36.304 INFO:teuthology.orchestra.run.smithi183.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2023-10-19T10:34:36.395 INFO:teuthology.orchestra.run.smithi183.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 645 kB/s | 86 kB 00:00 2023-10-19T10:34:36.420 INFO:teuthology.orchestra.run.smithi183.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2023-10-19T10:34:36.803 INFO:teuthology.orchestra.run.smithi183.stdout:(94/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 5.6 MB/s | 4.1 MB 00:00 2023-10-19T10:34:36.879 INFO:teuthology.orchestra.run.smithi183.stdout:(95/143): fmt-8.1.1-5.el9.x86_64.rpm 229 kB/s | 111 kB 00:00 2023-10-19T10:34:36.978 INFO:teuthology.orchestra.run.smithi183.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 244 kB/s | 24 kB 00:00 2023-10-19T10:34:37.004 INFO:teuthology.orchestra.run.smithi183.stdout:(97/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 529 kB/s | 309 kB 00:00 2023-10-19T10:34:37.104 INFO:teuthology.orchestra.run.smithi183.stdout:(98/143): liboath-2.6.7-2.el9.x86_64.rpm 387 kB/s | 49 kB 00:00 2023-10-19T10:34:37.179 INFO:teuthology.orchestra.run.smithi183.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 385 kB/s | 67 kB 00:00 2023-10-19T10:34:37.405 INFO:teuthology.orchestra.run.smithi183.stdout:(100/143): python3-asyncssh-2.13.2-3.el9.noarch 2.4 MB/s | 548 kB 00:00 2023-10-19T10:34:37.546 INFO:teuthology.orchestra.run.smithi183.stdout:(101/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2023-10-19T10:34:37.655 INFO:teuthology.orchestra.run.smithi183.stdout:(102/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.5 MB/s | 837 kB 00:00 2023-10-19T10:34:37.672 INFO:teuthology.orchestra.run.smithi183.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2023-10-19T10:34:37.780 INFO:teuthology.orchestra.run.smithi183.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2023-10-19T10:34:37.814 INFO:teuthology.orchestra.run.smithi183.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2023-10-19T10:34:37.939 INFO:teuthology.orchestra.run.smithi183.stdout:(106/143): python3-google-auth-2.23.3-1.el9.noa 1.5 MB/s | 197 kB 00:00 2023-10-19T10:34:38.058 INFO:teuthology.orchestra.run.smithi183.stdout:(107/143): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2023-10-19T10:34:38.090 INFO:teuthology.orchestra.run.smithi183.stdout:(108/143): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2023-10-19T10:34:38.092 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 970 kB/s | 19 MB 00:20 2023-10-19T10:34:38.182 INFO:teuthology.orchestra.run.smithi183.stdout:(109/143): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2023-10-19T10:34:38.199 INFO:teuthology.orchestra.run.smithi183.stdout:(110/143): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2023-10-19T10:34:38.316 INFO:teuthology.orchestra.run.smithi183.stdout:(111/143): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2023-10-19T10:34:38.333 INFO:teuthology.orchestra.run.smithi183.stdout:(112/143): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2023-10-19T10:34:38.441 INFO:teuthology.orchestra.run.smithi183.stdout:(113/143): python3-jwt+crypto-2.4.0-1.el9.noarc 72 kB/s | 9.0 kB 00:00 2023-10-19T10:34:38.466 INFO:teuthology.orchestra.run.smithi183.stdout:(114/143): python3-jwt-2.4.0-1.el9.noarch.rpm 306 kB/s | 41 kB 00:00 2023-10-19T10:34:38.583 INFO:teuthology.orchestra.run.smithi183.stdout:(115/143): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2023-10-19T10:34:38.684 INFO:teuthology.orchestra.run.smithi183.stdout:(116/143): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2023-10-19T10:34:38.751 INFO:teuthology.orchestra.run.smithi183.stdout:(117/143): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2023-10-19T10:34:38.792 INFO:teuthology.orchestra.run.smithi183.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 533 kB/s | 58 kB 00:00 2023-10-19T10:34:38.884 INFO:teuthology.orchestra.run.smithi183.stdout:(119/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-10-19T10:34:38.910 INFO:teuthology.orchestra.run.smithi183.stdout:(120/143): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2023-10-19T10:34:39.002 INFO:teuthology.orchestra.run.smithi183.stdout:(121/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 770 kB/s | 90 kB 00:00 2023-10-19T10:34:39.068 INFO:teuthology.orchestra.run.smithi183.stdout:(122/143): python3-repoze-lru-0.7-16.el9.noarch 194 kB/s | 31 kB 00:00 2023-10-19T10:34:39.135 INFO:teuthology.orchestra.run.smithi183.stdout:(123/143): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2023-10-19T10:34:39.194 INFO:teuthology.orchestra.run.smithi183.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2023-10-19T10:34:39.269 INFO:teuthology.orchestra.run.smithi183.stdout:(125/143): python3-tempora-5.0.0-2.el9.noarch.r 268 kB/s | 36 kB 00:00 2023-10-19T10:34:39.311 INFO:teuthology.orchestra.run.smithi183.stdout:(126/143): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2023-10-19T10:34:39.403 INFO:teuthology.orchestra.run.smithi183.stdout:(127/143): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-10-19T10:34:39.428 INFO:teuthology.orchestra.run.smithi183.stdout:(128/143): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2023-10-19T10:34:39.529 INFO:teuthology.orchestra.run.smithi183.stdout:(129/143): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2023-10-19T10:34:39.597 INFO:teuthology.orchestra.run.smithi183.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2023-10-19T10:34:39.713 INFO:teuthology.orchestra.run.smithi183.stdout:(131/143): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2023-10-19T10:34:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:(132/143): libarrow-9.0.0-9.el9.x86_64.rpm 1.4 MB/s | 4.4 MB 00:03 2023-10-19T10:34:40.073 INFO:teuthology.orchestra.run.smithi183.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2023-10-19T10:34:40.131 INFO:teuthology.orchestra.run.smithi183.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 68 kB 00:00 2023-10-19T10:34:40.273 INFO:teuthology.orchestra.run.smithi183.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 16 MB/s | 2.2 MB 00:00 2023-10-19T10:34:40.448 INFO:teuthology.orchestra.run.smithi183.stdout:(136/143): librados2-18.0.0-5610.g6f3e6b48.el9. 4.5 MB/s | 3.3 MB 00:00 2023-10-19T10:34:40.490 INFO:teuthology.orchestra.run.smithi183.stdout:(137/143): NetworkManager-libnm-1.45.4-1.el9.x8 8.2 MB/s | 1.8 MB 00:00 2023-10-19T10:34:40.507 INFO:teuthology.orchestra.run.smithi183.stdout:(138/143): NetworkManager-team-1.45.4-1.el9.x86 715 kB/s | 41 kB 00:00 2023-10-19T10:34:40.532 INFO:teuthology.orchestra.run.smithi183.stdout:(139/143): NetworkManager-tui-1.45.4-1.el9.x86_ 5.8 MB/s | 244 kB 00:00 2023-10-19T10:34:40.558 INFO:teuthology.orchestra.run.smithi183.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-10-19T10:34:40.574 INFO:teuthology.orchestra.run.smithi183.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 526 kB/s | 22 kB 00:00 2023-10-19T10:34:40.633 INFO:teuthology.orchestra.run.smithi183.stdout:(142/143): librbd1-18.0.0-5610.g6f3e6b48.el9.x8 4.2 MB/s | 3.0 MB 00:00 2023-10-19T10:34:40.658 INFO:teuthology.orchestra.run.smithi183.stdout:(143/143): xfsdump-3.1.12-4.el9.x86_64.rpm 3.1 MB/s | 321 kB 00:00 2023-10-19T10:34:40.662 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2023-10-19T10:34:40.662 INFO:teuthology.orchestra.run.smithi183.stdout:Total 14 MB/s | 198 MB 00:14 2023-10-19T10:34:40.798 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2023-10-19T10:34:40.921 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2023-10-19T10:34:40.922 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2023-10-19T10:34:42.671 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2023-10-19T10:34:42.672 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2023-10-19T10:34:42.698 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-19T10:34:45.211 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-19T10:34:45.212 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-19T10:34:45.223 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-19T10:34:45.224 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-19T10:34:45.225 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-19T10:34:45.226 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-19T10:34:45.228 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-19T10:34:45.230 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-19T10:34:45.231 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-19T10:34:45.306 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2023-10-19T10:34:45.324 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2023-10-19T10:34:45.324 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2023-10-19T10:34:45.324 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2023-10-19T10:34:45.325 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2023-10-19T10:34:45.325 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-19T10:34:45.325 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 6.5 k 2023-10-19T10:34:45.325 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 5.2 M 2023-10-19T10:34:45.325 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 849 k 2023-10-19T10:34:45.326 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 141 k 2023-10-19T10:34:45.326 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 1.5 M 2023-10-19T10:34:45.326 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 132 k 2023-10-19T10:34:45.326 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 1.7 M 2023-10-19T10:34:45.326 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 7.4 M 2023-10-19T10:34:45.327 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 47 k 2023-10-19T10:34:45.327 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 8.2 M 2023-10-19T10:34:45.327 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 40 M 2023-10-19T10:34:45.327 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 261 k 2023-10-19T10:34:45.327 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 225 k 2023-10-19T10:34:45.328 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 218 k 2023-10-19T10:34:45.328 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 18 k 2023-10-19T10:34:45.328 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-19T10:34:45.328 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-19T10:34:45.328 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-19T10:34:45.328 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 31 k 2023-10-19T10:34:45.329 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 666 k 2023-10-19T10:34:45.329 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 127 k 2023-10-19T10:34:45.329 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-19T10:34:45.329 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-19T10:34:45.329 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-19T10:34:45.330 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-19T10:34:45.330 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-19T10:34:45.330 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-19T10:34:45.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 162 k 2023-10-19T10:34:45.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 319 k 2023-10-19T10:34:45.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 298 k 2023-10-19T10:34:45.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 99 k 2023-10-19T10:34:45.331 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 85 k 2023-10-19T10:34:45.331 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 3.0 M 2023-10-19T10:34:45.331 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 169 k 2023-10-19T10:34:45.332 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-19T10:34:45.332 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2023-10-19T10:34:45.332 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-19T10:34:45.332 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-19T10:34:45.332 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-19T10:34:45.332 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-19T10:34:45.333 INFO:teuthology.orchestra.run.smithi050.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-19T10:34:45.333 INFO:teuthology.orchestra.run.smithi050.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-19T10:34:45.333 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-19T10:34:45.333 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 3.3 M 2023-10-19T10:34:45.333 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 3.0 M 2023-10-19T10:34:45.334 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-19T10:34:45.334 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2023-10-19T10:34:45.334 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-19T10:34:45.334 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-19T10:34:45.334 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 19 M 2023-10-19T10:34:45.335 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 23 k 2023-10-19T10:34:45.335 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 2.1 M 2023-10-19T10:34:45.335 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 242 k 2023-10-19T10:34:45.335 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 4.4 M 2023-10-19T10:34:45.335 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 18 M 2023-10-19T10:34:45.336 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5610.g6f3e6b48.el9 ceph-noarch 15 k 2023-10-19T10:34:45.336 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 24 k 2023-10-19T10:34:45.336 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-19T10:34:45.336 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-19T10:34:45.336 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-19T10:34:45.336 INFO:teuthology.orchestra.run.smithi050.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-19T10:34:45.337 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-19T10:34:45.337 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-19T10:34:45.337 INFO:teuthology.orchestra.run.smithi050.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-19T10:34:45.337 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-19T10:34:45.337 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-19T10:34:45.338 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 163 k 2023-10-19T10:34:45.338 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-19T10:34:45.338 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-19T10:34:45.338 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-19T10:34:45.338 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-19T10:34:45.339 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-19T10:34:45.339 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 472 k 2023-10-19T10:34:45.339 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-19T10:34:45.339 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 4.7 M 2023-10-19T10:34:45.339 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-19T10:34:45.339 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-19T10:34:45.340 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-19T10:34:45.340 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-19T10:34:45.340 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-19T10:34:45.340 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-19T10:34:45.340 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-19T10:34:45.341 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-19T10:34:45.341 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-19T10:34:45.341 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-19T10:34:45.341 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-19T10:34:45.341 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-19T10:34:45.342 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 45 k 2023-10-19T10:34:45.342 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:18.0.0-5610.g6f3e6b48.el9 ceph 127 k 2023-10-19T10:34:45.342 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-19T10:34:45.342 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-19T10:34:45.342 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-19T10:34:45.343 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-19T10:34:45.343 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-19T10:34:45.343 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-19T10:34:45.343 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-19T10:34:45.343 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-19T10:34:45.343 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-19T10:34:45.344 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-19T10:34:45.344 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-19T10:34:45.344 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-19T10:34:45.344 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-19T10:34:45.345 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-19T10:34:45.345 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-19T10:34:45.345 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-19T10:34:45.345 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-19T10:34:45.345 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-19T10:34:45.346 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-19T10:34:45.346 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-19T10:34:45.346 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-19T10:34:45.346 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-19T10:34:45.346 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-19T10:34:45.347 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-19T10:34:45.347 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-19T10:34:45.347 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-19T10:34:45.347 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-19T10:34:45.347 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-19T10:34:45.348 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-19T10:34:45.348 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-19T10:34:45.348 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-19T10:34:45.348 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-19T10:34:45.348 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-19T10:34:45.348 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-19T10:34:45.349 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-19T10:34:45.349 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-19T10:34:45.349 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-19T10:34:45.349 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-19T10:34:45.349 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-19T10:34:45.350 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-19T10:34:45.350 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-19T10:34:45.350 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-19T10:34:45.350 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-19T10:34:45.350 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-19T10:34:45.351 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-19T10:34:45.351 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-19T10:34:45.351 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-19T10:34:45.351 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-19T10:34:45.351 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-19T10:34:45.352 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-19T10:34:45.352 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-19T10:34:45.352 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-19T10:34:45.352 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-19T10:34:45.352 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-19T10:34:45.352 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2023-10-19T10:34:45.353 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-19T10:34:45.353 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:34:45.353 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2023-10-19T10:34:45.353 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2023-10-19T10:34:45.353 INFO:teuthology.orchestra.run.smithi050.stdout:Install 133 Packages 2023-10-19T10:34:45.354 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 10 Packages 2023-10-19T10:34:45.354 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:34:45.355 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 198 M 2023-10-19T10:34:45.355 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2023-10-19T10:34:45.974 INFO:teuthology.orchestra.run.smithi050.stdout:(1/143): ceph-18.0.0-5610.g6f3e6b48.el9.x86_64. 27 kB/s | 6.5 kB 00:00 2023-10-19T10:34:46.274 INFO:teuthology.orchestra.run.smithi050.stdout:(2/143): ceph-base-18.0.0-5610.g6f3e6b48.el9.x8 9.5 MB/s | 5.2 MB 00:00 2023-10-19T10:34:46.325 INFO:teuthology.orchestra.run.smithi050.stdout:(3/143): ceph-immutable-object-cache-18.0.0-561 2.8 MB/s | 141 kB 00:00 2023-10-19T10:34:46.475 INFO:teuthology.orchestra.run.smithi050.stdout:(4/143): ceph-mds-18.0.0-5610.g6f3e6b48.el9.x86 14 MB/s | 2.1 MB 00:00 2023-10-19T10:34:46.586 INFO:teuthology.orchestra.run.smithi050.stdout:(5/143): ceph-mgr-18.0.0-5610.g6f3e6b48.el9.x86 13 MB/s | 1.5 MB 00:00 2023-10-19T10:34:46.625 INFO:teuthology.orchestra.run.smithi050.stdout:(6/143): ceph-fuse-18.0.0-5610.g6f3e6b48.el9.x8 1.3 MB/s | 849 kB 00:00 2023-10-19T10:34:46.834 INFO:teuthology.orchestra.run.smithi050.stdout:(7/143): ceph-mon-18.0.0-5610.g6f3e6b48.el9.x86 17 MB/s | 4.4 MB 00:00 2023-10-19T10:34:47.267 INFO:teuthology.orchestra.run.smithi050.stdout:(8/143): ceph-radosgw-18.0.0-5610.g6f3e6b48.el9 19 MB/s | 8.2 MB 00:00 2023-10-19T10:34:47.309 INFO:teuthology.orchestra.run.smithi050.stdout:(9/143): ceph-selinux-18.0.0-5610.g6f3e6b48.el9 581 kB/s | 24 kB 00:00 2023-10-19T10:34:48.266 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2023-10-19T10:34:48.327 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-19T10:34:48.353 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-19T10:34:48.379 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-19T10:34:48.439 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-19T10:34:48.756 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-19T10:34:48.812 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 6/153 2023-10-19T10:34:48.927 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 6/153 2023-10-19T10:34:48.963 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 7/153 2023-10-19T10:34:49.055 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 7/153 2023-10-19T10:34:49.113 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:18.0.0-5610.g6f3e6b48.el9.x86_64 8/153 2023-10-19T10:34:49.153 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-19T10:34:49.304 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-19T10:34:49.348 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-19T10:34:49.376 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 12/153 2023-10-19T10:34:49.478 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 12/153 2023-10-19T10:34:49.502 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86 13/153 2023-10-19T10:34:49.564 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86 13/153 2023-10-19T10:34:49.627 INFO:teuthology.orchestra.run.smithi050.stdout:(10/143): ceph-test-18.0.0-5610.g6f3e6b48.el9.x 17 MB/s | 40 MB 00:02 2023-10-19T10:34:49.677 INFO:teuthology.orchestra.run.smithi050.stdout:(11/143): cephfs-mirror-18.0.0-5610.g6f3e6b48.e 4.3 MB/s | 218 kB 00:00 2023-10-19T10:34:49.730 INFO:teuthology.orchestra.run.smithi050.stdout:(12/143): libcephfs-devel-18.0.0-5610.g6f3e6b48 627 kB/s | 31 kB 00:00 2023-10-19T10:34:49.766 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-19T10:34:49.794 INFO:teuthology.orchestra.run.smithi050.stdout:(13/143): libcephfs2-18.0.0-5610.g6f3e6b48.el9. 9.8 MB/s | 666 kB 00:00 2023-10-19T10:34:49.852 INFO:teuthology.orchestra.run.smithi050.stdout:(14/143): libcephsqlite-18.0.0-5610.g6f3e6b48.e 2.7 MB/s | 163 kB 00:00 2023-10-19T10:34:49.890 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-19T10:34:49.903 INFO:teuthology.orchestra.run.smithi050.stdout:(15/143): librados-devel-18.0.0-5610.g6f3e6b48. 2.5 MB/s | 127 kB 00:00 2023-10-19T10:34:49.933 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-19T10:34:49.978 INFO:teuthology.orchestra.run.smithi050.stdout:(16/143): libradosstriper1-18.0.0-5610.g6f3e6b4 6.2 MB/s | 472 kB 00:00 2023-10-19T10:34:50.038 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-19T10:34:50.102 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-19T10:34:50.354 INFO:teuthology.orchestra.run.smithi050.stdout:(17/143): librgw2-18.0.0-5610.g6f3e6b48.el9.x86 12 MB/s | 4.7 MB 00:00 2023-10-19T10:34:50.395 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-19T10:34:50.431 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-19T10:34:50.466 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-19T10:34:50.529 INFO:teuthology.orchestra.run.smithi050.stdout:(18/143): ceph-osd-18.0.0-5610.g6f3e6b48.el9.x8 4.5 MB/s | 18 MB 00:03 2023-10-19T10:34:50.546 INFO:teuthology.orchestra.run.smithi050.stdout:(19/143): python3-ceph-argparse-18.0.0-5610.g6f 235 kB/s | 45 kB 00:00 2023-10-19T10:34:50.550 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-19T10:34:50.605 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-19T10:34:50.635 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-19T10:34:50.679 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-19T10:34:50.738 INFO:teuthology.orchestra.run.smithi050.stdout:(20/143): ceph-common-18.0.0-5610.g6f3e6b48.el9 3.8 MB/s | 19 MB 00:05 2023-10-19T10:34:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-19T10:34:50.763 INFO:teuthology.orchestra.run.smithi050.stdout:(21/143): python3-ceph-common-18.0.0-5610.g6f3e 542 kB/s | 127 kB 00:00 2023-10-19T10:34:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-19T10:34:50.788 INFO:teuthology.orchestra.run.smithi050.stdout:(22/143): python3-cephfs-18.0.0-5610.g6f3e6b48. 669 kB/s | 162 kB 00:00 2023-10-19T10:34:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-19T10:34:50.822 INFO:teuthology.orchestra.run.smithi050.stdout:(23/143): python3-rados-18.0.0-5610.g6f3e6b48.e 3.7 MB/s | 319 kB 00:00 2023-10-19T10:34:50.839 INFO:teuthology.orchestra.run.smithi050.stdout:(24/143): python3-rgw-18.0.0-5610.g6f3e6b48.el9 1.9 MB/s | 99 kB 00:00 2023-10-19T10:34:50.864 INFO:teuthology.orchestra.run.smithi050.stdout:(25/143): python3-rbd-18.0.0-5610.g6f3e6b48.el9 2.9 MB/s | 298 kB 00:00 2023-10-19T10:34:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-19T10:34:50.898 INFO:teuthology.orchestra.run.smithi050.stdout:(26/143): rbd-fuse-18.0.0-5610.g6f3e6b48.el9.x8 1.1 MB/s | 85 kB 00:00 2023-10-19T10:34:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:18.0.0-5610.g6f3e6b48.el9. 27/153 2023-10-19T10:34:50.957 INFO:teuthology.orchestra.run.smithi050.stdout:(27/143): rbd-nbd-18.0.0-5610.g6f3e6b48.el9.x86 1.8 MB/s | 169 kB 00:00 2023-10-19T10:34:50.968 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:18.0.0-5610.g6f3e6b48.el 28/153 2023-10-19T10:34:51.015 INFO:teuthology.orchestra.run.smithi050.stdout:(28/143): rbd-mirror-18.0.0-5610.g6f3e6b48.el9. 17 MB/s | 3.0 MB 00:00 2023-10-19T10:34:51.040 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:18.0.0-5610.g6f3e6b48.el9.x86_6 29/153 2023-10-19T10:34:51.041 INFO:teuthology.orchestra.run.smithi050.stdout:(29/143): ceph-grafana-dashboards-18.0.0-5610.g 163 kB/s | 23 kB 00:00 2023-10-19T10:34:51.066 INFO:teuthology.orchestra.run.smithi050.stdout:(30/143): ceph-mgr-cephadm-18.0.0-5610.g6f3e6b4 1.2 MB/s | 132 kB 00:00 2023-10-19T10:34:51.132 INFO:teuthology.orchestra.run.smithi050.stdout:(31/143): ceph-mgr-dashboard-18.0.0-5610.g6f3e6 15 MB/s | 1.7 MB 00:00 2023-10-19T10:34:51.166 INFO:teuthology.orchestra.run.smithi050.stdout:(32/143): ceph-mgr-modules-core-18.0.0-5610.g6f 2.4 MB/s | 242 kB 00:00 2023-10-19T10:34:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-19T10:34:51.191 INFO:teuthology.orchestra.run.smithi050.stdout:(33/143): ceph-mgr-rook-18.0.0-5610.g6f3e6b48.e 813 kB/s | 47 kB 00:00 2023-10-19T10:34:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-19T10:34:51.250 INFO:teuthology.orchestra.run.smithi050.stdout:(34/143): ceph-volume-18.0.0-5610.g6f3e6b48.el9 4.4 MB/s | 261 kB 00:00 2023-10-19T10:34:51.275 INFO:teuthology.orchestra.run.smithi050.stdout:(35/143): ceph-prometheus-alerts-18.0.0-5610.g6 135 kB/s | 15 kB 00:00 2023-10-19T10:34:51.326 INFO:teuthology.orchestra.run.smithi050.stdout:(36/143): cephfs-top-18.0.0-5610.g6f3e6b48.el9. 362 kB/s | 18 kB 00:00 2023-10-19T10:34:51.351 INFO:teuthology.orchestra.run.smithi050.stdout:(37/143): cephadm-18.0.0-5610.g6f3e6b48.el9.noa 2.2 MB/s | 225 kB 00:00 2023-10-19T10:34:51.559 INFO:teuthology.orchestra.run.smithi050.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 508 kB/s | 106 kB 00:00 2023-10-19T10:34:51.635 INFO:teuthology.orchestra.run.smithi050.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 920 kB 00:00 2023-10-19T10:34:51.660 INFO:teuthology.orchestra.run.smithi050.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 307 kB 00:00 2023-10-19T10:34:51.694 INFO:teuthology.orchestra.run.smithi050.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 545 kB/s | 32 kB 00:00 2023-10-19T10:34:51.750 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-19T10:34:51.823 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-19T10:34:51.860 INFO:teuthology.orchestra.run.smithi050.stdout:(42/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2023-10-19T10:34:51.861 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-19T10:34:51.886 INFO:teuthology.orchestra.run.smithi050.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 89 kB/s | 17 kB 00:00 2023-10-19T10:34:51.899 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_6 35/153 2023-10-19T10:34:51.911 INFO:teuthology.orchestra.run.smithi050.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2023-10-19T10:34:51.930 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-19T10:34:51.945 INFO:teuthology.orchestra.run.smithi050.stdout:(45/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2023-10-19T10:34:51.987 INFO:teuthology.orchestra.run.smithi050.stdout:(46/143): librdkafka-1.6.1-102.el9.x86_64.rpm 8.5 MB/s | 661 kB 00:00 2023-10-19T10:34:52.029 INFO:teuthology.orchestra.run.smithi050.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2023-10-19T10:34:52.036 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-19T10:34:52.054 INFO:teuthology.orchestra.run.smithi050.stdout:(48/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 693 kB/s | 46 kB 00:00 2023-10-19T10:34:52.076 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-19T10:34:52.104 INFO:teuthology.orchestra.run.smithi050.stdout:(49/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-10-19T10:34:52.130 INFO:teuthology.orchestra.run.smithi050.stdout:(50/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2023-10-19T10:34:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-19T10:34:52.163 INFO:teuthology.orchestra.run.smithi050.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 584 kB/s | 34 kB 00:00 2023-10-19T10:34:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-19T10:34:52.243 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-19T10:34:52.371 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-19T10:34:52.389 INFO:teuthology.orchestra.run.smithi050.stdout:(52/143): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2023-10-19T10:34:52.418 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-19T10:34:52.439 INFO:teuthology.orchestra.run.smithi050.stdout:(53/143): postgresql-13.11-1.el9.x86_64.rpm 5.5 MB/s | 1.5 MB 00:00 2023-10-19T10:34:52.463 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-19T10:34:52.472 INFO:teuthology.orchestra.run.smithi050.stdout:(54/143): postgresql-contrib-13.11-1.el9.x86_64 9.9 MB/s | 811 kB 00:00 2023-10-19T10:34:52.498 INFO:teuthology.orchestra.run.smithi050.stdout:(55/143): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2023-10-19T10:34:52.505 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-19T10:34:52.598 INFO:teuthology.orchestra.run.smithi050.stdout:(56/143): ceph-mgr-diskprediction-local-18.0.0- 4.7 MB/s | 7.4 MB 00:01 2023-10-19T10:34:52.639 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-19T10:34:52.682 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-19T10:34:52.690 INFO:teuthology.orchestra.run.smithi050.stdout:(57/143): postgresql-server-13.11-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2023-10-19T10:34:52.724 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-19T10:34:52.840 INFO:teuthology.orchestra.run.smithi050.stdout:(58/143): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-10-19T10:34:52.841 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-19T10:34:52.866 INFO:teuthology.orchestra.run.smithi050.stdout:(59/143): python3-devel-3.9.17-2.el9.x86_64.rpm 776 kB/s | 205 kB 00:00 2023-10-19T10:34:52.891 INFO:teuthology.orchestra.run.smithi050.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.1 MB/s | 228 kB 00:00 2023-10-19T10:34:52.892 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/153 2023-10-19T10:34:52.916 INFO:teuthology.orchestra.run.smithi050.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 328 kB/s | 24 kB 00:00 2023-10-19T10:34:52.932 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-19T10:34:52.942 INFO:teuthology.orchestra.run.smithi050.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2023-10-19T10:34:52.964 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-19T10:34:52.967 INFO:teuthology.orchestra.run.smithi050.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-10-19T10:34:52.992 INFO:teuthology.orchestra.run.smithi050.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 418 kB/s | 32 kB 00:00 2023-10-19T10:34:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-19T10:34:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-19T10:34:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-19T10:34:53.152 INFO:teuthology.orchestra.run.smithi050.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2023-10-19T10:34:53.176 INFO:teuthology.orchestra.run.smithi050.stdout:(66/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2023-10-19T10:34:53.197 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-19T10:34:53.201 INFO:teuthology.orchestra.run.smithi050.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 844 kB/s | 174 kB 00:00 2023-10-19T10:34:53.226 INFO:teuthology.orchestra.run.smithi050.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 554 kB/s | 41 kB 00:00 2023-10-19T10:34:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-19T10:34:53.252 INFO:teuthology.orchestra.run.smithi050.stdout:(69/143): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2023-10-19T10:34:53.277 INFO:teuthology.orchestra.run.smithi050.stdout:(70/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-10-19T10:34:53.292 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-19T10:34:53.302 INFO:teuthology.orchestra.run.smithi050.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 565 kB/s | 43 kB 00:00 2023-10-19T10:34:53.348 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-19T10:34:53.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-19T10:34:53.569 INFO:teuthology.orchestra.run.smithi050.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 132 kB/s | 39 kB 00:00 2023-10-19T10:34:53.581 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-19T10:34:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-19T10:34:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-19T10:34:53.781 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-19T10:34:53.814 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-19T10:34:53.848 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-19T10:34:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-19T10:34:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-19T10:34:53.920 INFO:teuthology.orchestra.run.smithi050.stdout:(73/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2023-10-19T10:34:53.945 INFO:teuthology.orchestra.run.smithi050.stdout:(74/143): socat-1.7.4.1-5.el9.x86_64.rpm 468 kB/s | 300 kB 00:00 2023-10-19T10:34:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-19T10:34:53.987 INFO:teuthology.orchestra.run.smithi050.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 136 kB/s | 56 kB 00:00 2023-10-19T10:34:54.001 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-19T10:34:54.012 INFO:teuthology.orchestra.run.smithi050.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 686 kB/s | 63 kB 00:00 2023-10-19T10:34:54.031 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-19T10:34:54.037 INFO:teuthology.orchestra.run.smithi050.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2023-10-19T10:34:54.071 INFO:teuthology.orchestra.run.smithi050.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-10-19T10:34:54.096 INFO:teuthology.orchestra.run.smithi050.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2023-10-19T10:34:54.115 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-19T10:34:54.121 INFO:teuthology.orchestra.run.smithi050.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2023-10-19T10:34:54.147 INFO:teuthology.orchestra.run.smithi050.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2023-10-19T10:34:54.149 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-19T10:34:54.172 INFO:teuthology.orchestra.run.smithi050.stdout:(82/143): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2023-10-19T10:34:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-19T10:34:54.197 INFO:teuthology.orchestra.run.smithi050.stdout:(83/143): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2023-10-19T10:34:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-19T10:34:54.222 INFO:teuthology.orchestra.run.smithi050.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 2.4 MB/s | 188 kB 00:00 2023-10-19T10:34:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-19T10:34:54.339 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-19T10:34:54.378 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-19T10:34:54.406 INFO:teuthology.orchestra.run.smithi050.stdout:(85/143): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2023-10-19T10:34:54.480 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-19T10:34:54.528 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-19T10:34:54.898 INFO:teuthology.orchestra.run.smithi050.stdout:(86/143): elfutils-libelf-devel-0.189-3.el9.x86 33 kB/s | 22 kB 00:00 2023-10-19T10:34:55.260 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-19T10:34:55.315 INFO:teuthology.orchestra.run.smithi050.stdout:(87/143): automake-1.16.2-8.el9.noarch.rpm 631 kB/s | 689 kB 00:01 2023-10-19T10:34:55.317 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-19T10:34:55.340 INFO:teuthology.orchestra.run.smithi050.stdout:(88/143): jq-1.6-15.el9.x86_64.rpm 200 kB/s | 187 kB 00:00 2023-10-19T10:34:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-19T10:34:55.450 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-19T10:34:55.493 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-19T10:34:55.657 INFO:teuthology.orchestra.run.smithi050.stdout:(89/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 60 kB/s | 45 kB 00:00 2023-10-19T10:34:55.669 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-19T10:34:55.692 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-19T10:34:55.769 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-19T10:34:56.010 INFO:teuthology.orchestra.run.smithi050.stdout:(90/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 37 kB/s | 26 kB 00:00 2023-10-19T10:34:56.420 INFO:teuthology.orchestra.run.smithi050.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 67 kB/s | 51 kB 00:00 2023-10-19T10:34:56.420 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-19T10:34:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-19T10:34:56.686 INFO:teuthology.orchestra.run.smithi050.stdout:(92/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 3.1 MB/s | 4.1 MB 00:01 2023-10-19T10:34:56.796 INFO:teuthology.orchestra.run.smithi050.stdout:(93/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 110 kB/s | 86 kB 00:00 2023-10-19T10:34:56.945 INFO:teuthology.orchestra.run.smithi050.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 479 kB/s | 111 kB 00:00 2023-10-19T10:34:56.978 INFO:teuthology.orchestra.run.smithi050.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.7 MB/s | 309 kB 00:00 2023-10-19T10:34:57.029 INFO:teuthology.orchestra.run.smithi050.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 423 kB/s | 24 kB 00:00 2023-10-19T10:34:57.171 INFO:teuthology.orchestra.run.smithi050.stdout:(97/143): libarrow-9.0.0-9.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2023-10-19T10:34:57.196 INFO:teuthology.orchestra.run.smithi050.stdout:(98/143): liboath-2.6.7-2.el9.x86_64.rpm 292 kB/s | 49 kB 00:00 2023-10-19T10:34:57.222 INFO:teuthology.orchestra.run.smithi050.stdout:(99/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 56 kB/s | 45 kB 00:00 2023-10-19T10:34:57.247 INFO:teuthology.orchestra.run.smithi050.stdout:(100/143): libunwind-1.6.2-1.el9.x86_64.rpm 895 kB/s | 67 kB 00:00 2023-10-19T10:34:57.281 INFO:teuthology.orchestra.run.smithi050.stdout:(101/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 10 MB/s | 837 kB 00:00 2023-10-19T10:34:57.306 INFO:teuthology.orchestra.run.smithi050.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 763 kB/s | 43 kB 00:00 2023-10-19T10:34:57.331 INFO:teuthology.orchestra.run.smithi050.stdout:(103/143): python3-asyncssh-2.13.2-3.el9.noarch 6.5 MB/s | 548 kB 00:00 2023-10-19T10:34:57.357 INFO:teuthology.orchestra.run.smithi050.stdout:(104/143): python3-cachetools-4.2.4-1.el9.noarc 426 kB/s | 32 kB 00:00 2023-10-19T10:34:57.381 INFO:teuthology.orchestra.run.smithi050.stdout:(105/143): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2023-10-19T10:34:57.406 INFO:teuthology.orchestra.run.smithi050.stdout:(106/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2023-10-19T10:34:57.433 INFO:teuthology.orchestra.run.smithi050.stdout:(107/143): python3-google-auth-2.23.3-1.el9.noa 3.8 MB/s | 197 kB 00:00 2023-10-19T10:34:57.466 INFO:teuthology.orchestra.run.smithi050.stdout:(108/143): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2023-10-19T10:34:57.492 INFO:teuthology.orchestra.run.smithi050.stdout:(109/143): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2023-10-19T10:34:57.516 INFO:teuthology.orchestra.run.smithi050.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2023-10-19T10:34:57.542 INFO:teuthology.orchestra.run.smithi050.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2023-10-19T10:34:57.559 INFO:teuthology.orchestra.run.smithi050.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 466 kB/s | 19 kB 00:00 2023-10-19T10:34:57.576 INFO:teuthology.orchestra.run.smithi050.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2023-10-19T10:34:57.592 INFO:teuthology.orchestra.run.smithi050.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 180 kB/s | 9.0 kB 00:00 2023-10-19T10:34:57.618 INFO:teuthology.orchestra.run.smithi050.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 982 kB/s | 41 kB 00:00 2023-10-19T10:34:57.660 INFO:teuthology.orchestra.run.smithi050.stdout:(116/143): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2023-10-19T10:34:57.676 INFO:teuthology.orchestra.run.smithi050.stdout:(117/143): python3-logutils-0.3.5-21.el9.noarch 551 kB/s | 46 kB 00:00 2023-10-19T10:34:57.702 INFO:teuthology.orchestra.run.smithi050.stdout:(118/143): python3-more-itertools-8.12.0-2.el9. 966 kB/s | 79 kB 00:00 2023-10-19T10:34:57.718 INFO:teuthology.orchestra.run.smithi050.stdout:(119/143): python3-natsort-7.1.1-5.el9.noarch.r 992 kB/s | 58 kB 00:00 2023-10-19T10:34:57.744 INFO:teuthology.orchestra.run.smithi050.stdout:(120/143): python3-portend-3.1.0-2.el9.noarch.r 394 kB/s | 16 kB 00:00 2023-10-19T10:34:57.766 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-19T10:34:57.769 INFO:teuthology.orchestra.run.smithi050.stdout:(121/143): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2023-10-19T10:34:57.794 INFO:teuthology.orchestra.run.smithi050.stdout:(122/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2023-10-19T10:34:57.820 INFO:teuthology.orchestra.run.smithi050.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2023-10-19T10:34:57.843 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-19T10:34:57.845 INFO:teuthology.orchestra.run.smithi050.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2023-10-19T10:34:57.870 INFO:teuthology.orchestra.run.smithi050.stdout:(125/143): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2023-10-19T10:34:57.875 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-19T10:34:57.895 INFO:teuthology.orchestra.run.smithi050.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2023-10-19T10:34:57.906 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-19T10:34:57.920 INFO:teuthology.orchestra.run.smithi050.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2023-10-19T10:34:57.934 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-19T10:34:57.946 INFO:teuthology.orchestra.run.smithi050.stdout:(128/143): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2023-10-19T10:34:57.971 INFO:teuthology.orchestra.run.smithi050.stdout:(129/143): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2023-10-19T10:34:58.013 INFO:teuthology.orchestra.run.smithi050.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2023-10-19T10:34:58.038 INFO:teuthology.orchestra.run.smithi050.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2023-10-19T10:34:58.064 INFO:teuthology.orchestra.run.smithi050.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2023-10-19T10:34:58.114 INFO:teuthology.orchestra.run.smithi050.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2023-10-19T10:34:58.148 INFO:teuthology.orchestra.run.smithi050.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-10-19T10:34:58.233 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-19T10:34:58.263 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 97/153 2023-10-19T10:34:58.281 INFO:teuthology.orchestra.run.smithi050.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 17 MB/s | 2.2 MB 00:00 2023-10-19T10:34:58.344 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 97/153 2023-10-19T10:34:58.381 INFO:teuthology.orchestra.run.smithi050.stdout:(136/143): librbd1-18.0.0-5610.g6f3e6b48.el9.x8 9.6 MB/s | 3.0 MB 00:00 2023-10-19T10:34:58.384 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 98/153 2023-10-19T10:34:58.421 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-19T10:34:58.423 INFO:teuthology.orchestra.run.smithi050.stdout:(137/143): NetworkManager-libnm-1.45.4-1.el9.x8 13 MB/s | 1.8 MB 00:00 2023-10-19T10:34:58.449 INFO:teuthology.orchestra.run.smithi050.stdout:(138/143): NetworkManager-team-1.45.4-1.el9.x86 620 kB/s | 41 kB 00:00 2023-10-19T10:34:58.474 INFO:teuthology.orchestra.run.smithi050.stdout:(139/143): NetworkManager-tui-1.45.4-1.el9.x86_ 4.7 MB/s | 244 kB 00:00 2023-10-19T10:34:58.491 INFO:teuthology.orchestra.run.smithi050.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2023-10-19T10:34:58.516 INFO:teuthology.orchestra.run.smithi050.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 527 kB/s | 22 kB 00:00 2023-10-19T10:34:58.542 INFO:teuthology.orchestra.run.smithi050.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-10-19T10:34:58.608 INFO:teuthology.orchestra.run.smithi050.stdout:(143/143): librados2-18.0.0-5610.g6f3e6b48.el9. 5.8 MB/s | 3.3 MB 00:00 2023-10-19T10:34:58.615 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2023-10-19T10:34:58.615 INFO:teuthology.orchestra.run.smithi050.stdout:Total 15 MB/s | 198 MB 00:13 2023-10-19T10:34:58.755 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2023-10-19T10:34:58.835 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-19T10:34:58.866 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 101/153 2023-10-19T10:34:58.895 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2023-10-19T10:34:58.895 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2023-10-19T10:34:58.920 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 101/153 2023-10-19T10:34:58.942 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 102/153 2023-10-19T10:35:00.741 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2023-10-19T10:35:00.741 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2023-10-19T10:35:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 103/153 2023-10-19T10:35:01.097 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 103/153 2023-10-19T10:35:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 103/153 2023-10-19T10:35:01.684 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 104/153 2023-10-19T10:35:01.771 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 104/153 2023-10-19T10:35:01.771 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-19T10:35:01.771 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-19T10:35:01.771 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:01.810 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 105/153 2023-10-19T10:35:06.344 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2023-10-19T10:35:06.425 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-19T10:35:06.458 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-19T10:35:06.485 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-19T10:35:06.544 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-19T10:35:06.886 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-19T10:35:06.943 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 6/153 2023-10-19T10:35:07.061 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 6/153 2023-10-19T10:35:07.085 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 7/153 2023-10-19T10:35:07.178 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 7/153 2023-10-19T10:35:07.226 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:18.0.0-5610.g6f3e6b48.el9.x86_64 8/153 2023-10-19T10:35:07.265 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-19T10:35:07.435 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-19T10:35:07.479 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-19T10:35:07.507 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 12/153 2023-10-19T10:35:07.602 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 12/153 2023-10-19T10:35:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86 13/153 2023-10-19T10:35:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86 13/153 2023-10-19T10:35:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-19T10:35:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-19T10:35:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-19T10:35:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-19T10:35:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-19T10:35:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-19T10:35:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-19T10:35:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-19T10:35:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-19T10:35:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-19T10:35:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-19T10:35:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-19T10:35:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-19T10:35:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-19T10:35:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-19T10:35:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-19T10:35:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:18.0.0-5610.g6f3e6b48.el9. 27/153 2023-10-19T10:35:09.098 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:18.0.0-5610.g6f3e6b48.el 28/153 2023-10-19T10:35:09.176 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:18.0.0-5610.g6f3e6b48.el9.x86_6 29/153 2023-10-19T10:35:09.322 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-19T10:35:09.377 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-19T10:35:09.953 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-19T10:35:10.041 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-19T10:35:10.109 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-19T10:35:10.147 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_6 35/153 2023-10-19T10:35:10.177 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-19T10:35:10.279 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-19T10:35:10.322 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-19T10:35:10.379 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-19T10:35:10.454 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-19T10:35:10.489 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-19T10:35:10.526 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-19T10:35:10.574 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-19T10:35:10.619 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-19T10:35:10.660 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-19T10:35:10.806 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-19T10:35:10.845 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-19T10:35:10.887 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-19T10:35:10.986 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-19T10:35:11.020 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/153 2023-10-19T10:35:11.061 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-19T10:35:11.093 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-19T10:35:11.121 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-19T10:35:11.154 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-19T10:35:11.182 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-19T10:35:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-19T10:35:11.315 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-19T10:35:11.374 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-19T10:35:11.438 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-19T10:35:11.642 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-19T10:35:11.694 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-19T10:35:11.840 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-19T10:35:11.889 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-19T10:35:11.917 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-19T10:35:11.959 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-19T10:35:11.992 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-19T10:35:12.024 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-19T10:35:12.072 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-19T10:35:12.121 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-19T10:35:12.155 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-19T10:35:12.185 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-19T10:35:12.265 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-19T10:35:12.295 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-19T10:35:12.328 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-19T10:35:12.371 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-19T10:35:12.404 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-19T10:35:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-19T10:35:12.532 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-19T10:35:12.642 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-19T10:35:12.702 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-19T10:35:12.892 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 105/153 2023-10-19T10:35:12.893 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2023-10-19T10:35:12.893 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2023-10-19T10:35:12.893 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2023-10-19T10:35:12.893 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2023-10-19T10:35:12.894 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2023-10-19T10:35:12.894 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2023-10-19T10:35:12.894 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2023-10-19T10:35:12.894 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:13.054 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 106/153 2023-10-19T10:35:13.068 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 106/153 2023-10-19T10:35:13.069 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:13.069 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-19T10:35:13.069 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:13.487 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-19T10:35:13.548 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-19T10:35:13.583 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-19T10:35:13.701 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-19T10:35:13.731 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-19T10:35:13.915 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-19T10:35:13.949 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-19T10:35:14.031 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-19T10:35:14.633 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 107/153 2023-10-19T10:35:14.658 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 107/153 2023-10-19T10:35:14.659 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:14.659 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-19T10:35:14.659 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-19T10:35:14.659 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-19T10:35:14.660 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:14.746 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-19T10:35:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 108/153 2023-10-19T10:35:14.873 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-19T10:35:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 108/153 2023-10-19T10:35:14.881 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:14.881 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-19T10:35:14.881 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-19T10:35:14.882 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-19T10:35:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 109/153 2023-10-19T10:35:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 109/153 2023-10-19T10:35:15.349 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:15.351 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-19T10:35:15.351 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-19T10:35:15.351 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-19T10:35:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 110/153 2023-10-19T10:35:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 110/153 2023-10-19T10:35:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5610.g6f3e6b48.e 111/153 2023-10-19T10:35:15.931 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5610.g6f3e6b48. 112/153 2023-10-19T10:35:15.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noa 113/153 2023-10-19T10:35:16.177 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noa 113/153 2023-10-19T10:35:16.207 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.n 114/153 2023-10-19T10:35:16.213 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-19T10:35:16.305 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-19T10:35:16.340 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-19T10:35:16.367 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-19T10:35:16.395 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-19T10:35:16.713 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-19T10:35:16.741 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 97/153 2023-10-19T10:35:16.823 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 97/153 2023-10-19T10:35:16.854 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 98/153 2023-10-19T10:35:16.890 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-19T10:35:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.n 114/153 2023-10-19T10:35:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3 115/153 2023-10-19T10:35:17.297 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3 115/153 2023-10-19T10:35:17.332 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-19T10:35:17.360 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 101/153 2023-10-19T10:35:17.431 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 101/153 2023-10-19T10:35:17.446 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5610.g6f3e6b48.el 116/153 2023-10-19T10:35:17.462 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 102/153 2023-10-19T10:35:17.476 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 117/153 2023-10-19T10:35:17.498 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 117/153 2023-10-19T10:35:17.499 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:17.499 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-19T10:35:17.499 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-19T10:35:17.499 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-19T10:35:17.500 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:17.544 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 118/153 2023-10-19T10:35:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 118/153 2023-10-19T10:35:18.309 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:18.0.0-5610.g6f3e6b48.el9.x86_64 119/153 2023-10-19T10:35:18.354 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 120/153 2023-10-19T10:35:18.375 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 120/153 2023-10-19T10:35:18.376 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:18.376 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-19T10:35:18.376 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-19T10:35:18.376 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-19T10:35:18.376 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:18.412 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6 121/153 2023-10-19T10:35:18.434 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6 121/153 2023-10-19T10:35:18.434 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:18.434 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-19T10:35:18.435 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:18.479 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 122/153 2023-10-19T10:35:18.501 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 122/153 2023-10-19T10:35:18.502 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:18.502 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-19T10:35:18.502 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-19T10:35:18.502 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-19T10:35:18.503 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 123/153 2023-10-19T10:35:18.835 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 123/153 2023-10-19T10:35:18.836 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:18.836 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-19T10:35:18.836 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-19T10:35:18.836 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-19T10:35:18.837 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:19.678 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 103/153 2023-10-19T10:35:19.708 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 103/153 2023-10-19T10:35:20.271 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 103/153 2023-10-19T10:35:20.320 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 104/153 2023-10-19T10:35:20.412 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 104/153 2023-10-19T10:35:20.412 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-19T10:35:20.412 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-19T10:35:20.412 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:20.446 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 105/153 2023-10-19T10:35:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:18.0.0-5610.g6f3e6b48.el9.x86_64 124/153 2023-10-19T10:35:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 125/153 2023-10-19T10:35:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 126/153 2023-10-19T10:35:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-19T10:35:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-19T10:35:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 129/153 2023-10-19T10:35:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-19T10:35:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-19T10:35:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_ 132/153 2023-10-19T10:35:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-19T10:35:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-19T10:35:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-19T10:35:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-19T10:35:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-19T10:35:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-19T10:35:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-19T10:35:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:18.0.0-5610.g6f3e6b48.el9.noarch 138/153 2023-10-19T10:35:24.828 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-19T10:35:24.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-19T10:35:24.935 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-19T10:35:25.016 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-19T10:35:25.048 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-19T10:35:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-19T10:35:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-19T10:35:25.166 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-19T10:35:25.191 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-19T10:35:25.216 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-19T10:35:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-19T10:35:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-19T10:35:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-19T10:35:25.428 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-19T10:35:25.428 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-19T10:35:25.503 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-19T10:35:25.544 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-19T10:35:25.545 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-19T10:35:25.612 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-19T10:35:25.677 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-19T10:35:30.014 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-19T10:35:30.014 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:18.0.0-5610.g6f3e6b48.el9.x86_64 1/153 2023-10-19T10:35:30.015 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2/153 2023-10-19T10:35:30.015 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 3/153 2023-10-19T10:35:30.015 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 4/153 2023-10-19T10:35:30.015 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6 5/153 2023-10-19T10:35:30.015 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 6/153 2023-10-19T10:35:30.016 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 7/153 2023-10-19T10:35:30.016 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 8/153 2023-10-19T10:35:30.016 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 9/153 2023-10-19T10:35:30.016 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 10/153 2023-10-19T10:35:30.016 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 11/153 2023-10-19T10:35:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:18.0.0-5610.g6f3e6b48.el9.x86_64 12/153 2023-10-19T10:35:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 13/153 2023-10-19T10:35:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_ 14/153 2023-10-19T10:35:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 15/153 2023-10-19T10:35:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 16/153 2023-10-19T10:35:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_6 17/153 2023-10-19T10:35:30.018 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86 18/153 2023-10-19T10:35:30.018 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 19/153 2023-10-19T10:35:30.018 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5610.g6f3e6b48.el 20/153 2023-10-19T10:35:30.018 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:18.0.0-5610.g6f3e6b48.el9. 21/153 2023-10-19T10:35:30.018 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:18.0.0-5610.g6f3e6b48.el9.x86_6 22/153 2023-10-19T10:35:30.019 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:18.0.0-5610.g6f3e6b48.el9.x86_64 23/153 2023-10-19T10:35:30.019 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 24/153 2023-10-19T10:35:30.019 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 25/153 2023-10-19T10:35:30.019 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 26/153 2023-10-19T10:35:30.019 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 27/153 2023-10-19T10:35:30.030 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 28/153 2023-10-19T10:35:30.031 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5610.g6f3e6b48. 29/153 2023-10-19T10:35:30.031 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noa 30/153 2023-10-19T10:35:30.031 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.n 31/153 2023-10-19T10:35:30.031 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3 32/153 2023-10-19T10:35:30.031 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5610.g6f3e6b48.el 33/153 2023-10-19T10:35:30.032 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 34/153 2023-10-19T10:35:30.032 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5610.g6f3e6b48.e 35/153 2023-10-19T10:35:30.032 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 36/153 2023-10-19T10:35:30.032 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 37/153 2023-10-19T10:35:30.033 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:18.0.0-5610.g6f3e6b48.el9.noarch 38/153 2023-10-19T10:35:30.033 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-19T10:35:30.033 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-19T10:35:30.033 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-19T10:35:30.034 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-19T10:35:30.034 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-19T10:35:30.034 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-19T10:35:30.034 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-19T10:35:30.034 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-19T10:35:30.035 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-19T10:35:30.035 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-19T10:35:30.035 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-19T10:35:30.036 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-19T10:35:30.036 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-19T10:35:30.036 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-19T10:35:30.036 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-19T10:35:30.037 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-19T10:35:30.037 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-19T10:35:30.037 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-19T10:35:30.037 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-19T10:35:30.037 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-19T10:35:30.038 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-19T10:35:30.038 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-19T10:35:30.038 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-19T10:35:30.038 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-19T10:35:30.039 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-19T10:35:30.041 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-19T10:35:30.041 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-19T10:35:30.041 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-19T10:35:30.042 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-19T10:35:30.042 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-19T10:35:30.042 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-19T10:35:30.042 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-19T10:35:30.042 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-19T10:35:30.043 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-19T10:35:30.043 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-19T10:35:30.043 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-19T10:35:30.043 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-19T10:35:30.043 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-19T10:35:30.044 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-19T10:35:30.044 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-19T10:35:30.044 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-19T10:35:30.044 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-19T10:35:30.044 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-19T10:35:30.045 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-19T10:35:30.045 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-19T10:35:30.045 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-19T10:35:30.046 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-19T10:35:30.046 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-19T10:35:30.046 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-19T10:35:30.046 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-19T10:35:30.047 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-19T10:35:30.047 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-19T10:35:30.047 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-19T10:35:30.047 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-19T10:35:30.047 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-19T10:35:30.048 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-19T10:35:30.048 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-19T10:35:30.048 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-19T10:35:30.048 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-19T10:35:30.048 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-19T10:35:30.049 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-19T10:35:30.049 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-19T10:35:30.049 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-19T10:35:30.049 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-19T10:35:30.049 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-19T10:35:30.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-19T10:35:30.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-19T10:35:30.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-19T10:35:30.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-19T10:35:30.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 108/153 2023-10-19T10:35:30.051 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-19T10:35:30.051 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-19T10:35:30.051 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-19T10:35:30.051 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-19T10:35:30.051 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-19T10:35:30.052 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-19T10:35:30.052 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-19T10:35:30.052 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-19T10:35:30.052 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-19T10:35:30.052 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-19T10:35:30.052 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-19T10:35:30.053 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-19T10:35:30.053 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-19T10:35:30.053 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-19T10:35:30.053 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-19T10:35:30.053 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-19T10:35:30.054 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-19T10:35:30.054 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-19T10:35:30.054 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-19T10:35:30.054 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-19T10:35:30.054 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-19T10:35:30.055 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-19T10:35:30.055 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-19T10:35:30.055 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-19T10:35:30.055 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-19T10:35:30.055 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 134/153 2023-10-19T10:35:30.056 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-19T10:35:30.056 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 136/153 2023-10-19T10:35:30.056 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-19T10:35:30.056 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-19T10:35:30.056 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-19T10:35:30.056 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-19T10:35:30.057 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-19T10:35:30.057 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-19T10:35:30.057 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-19T10:35:30.057 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-19T10:35:30.057 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-19T10:35:30.058 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-19T10:35:30.058 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-19T10:35:30.058 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-19T10:35:30.058 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-19T10:35:30.058 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-19T10:35:30.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-19T10:35:30.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-19T10:35:32.627 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 105/153 2023-10-19T10:35:32.628 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2023-10-19T10:35:32.628 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2023-10-19T10:35:32.628 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2023-10-19T10:35:32.628 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2023-10-19T10:35:32.629 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2023-10-19T10:35:32.629 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2023-10-19T10:35:32.629 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2023-10-19T10:35:32.629 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:32.800 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 106/153 2023-10-19T10:35:32.817 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 106/153 2023-10-19T10:35:32.817 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:32.818 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-19T10:35:32.818 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-19T10:35:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:32.869 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2023-10-19T10:35:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: acl-2.3.1-4.el9.x86_64 2023-10-19T10:35:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-19T10:35:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-19T10:35:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-19T10:35:32.871 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2023-10-19T10:35:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2023-10-19T10:35:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2023-10-19T10:35:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-19T10:35:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-19T10:35:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2023-10-19T10:35:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-19T10:35:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-19T10:35:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-19T10:35:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-19T10:35:32.878 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-19T10:35:32.878 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-19T10:35:32.878 INFO:teuthology.orchestra.run.smithi183.stdout: jq-1.6-15.el9.x86_64 2023-10-19T10:35:32.878 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-19T10:35:32.878 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-19T10:35:32.878 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-19T10:35:32.879 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.879 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.879 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.879 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-19T10:35:32.879 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-19T10:35:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-19T10:35:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-19T10:35:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-19T10:35:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-19T10:35:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-19T10:35:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-19T10:35:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-19T10:35:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-19T10:35:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-19T10:35:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-19T10:35:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.11-1.el9.x86_64 2023-10-19T10:35:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-19T10:35:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-19T10:35:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-19T10:35:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-19T10:35:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-19T10:35:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-19T10:35:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-19T10:35:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-19T10:35:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-19T10:35:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-19T10:35:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-19T10:35:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-19T10:35:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-19T10:35:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-19T10:35:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-19T10:35:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-19T10:35:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-19T10:35:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-19T10:35:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-19T10:35:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-19T10:35:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-19T10:35:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-19T10:35:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-19T10:35:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-19T10:35:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-19T10:35:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-19T10:35:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-19T10:35:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-19T10:35:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-19T10:35:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-19T10:35:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-19T10:35:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-19T10:35:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-19T10:35:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-19T10:35:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2023-10-19T10:35:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-19T10:35:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-19T10:35:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-19T10:35:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-19T10:35:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-19T10:35:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-19T10:35:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-19T10:35:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-19T10:35:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-19T10:35:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-19T10:35:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-19T10:35:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-19T10:35:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-19T10:35:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-19T10:35:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-19T10:35:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-19T10:35:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-19T10:35:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-19T10:35:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-19T10:35:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-19T10:35:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-19T10:35:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2023-10-19T10:35:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-19T10:35:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-19T10:35:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-19T10:35:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-19T10:35:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-19T10:35:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-19T10:35:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:32.896 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-10-19T10:35:33.300 DEBUG:teuthology.parallel:result is None 2023-10-19T10:35:34.419 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 107/153 2023-10-19T10:35:34.447 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 107/153 2023-10-19T10:35:34.447 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:34.447 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-19T10:35:34.448 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-19T10:35:34.448 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-19T10:35:34.448 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:34.661 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 108/153 2023-10-19T10:35:34.685 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 108/153 2023-10-19T10:35:34.686 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:34.686 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-19T10:35:34.686 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-19T10:35:34.686 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-19T10:35:34.687 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:35.163 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 109/153 2023-10-19T10:35:35.187 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 109/153 2023-10-19T10:35:35.187 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:35.187 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-19T10:35:35.187 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-19T10:35:35.188 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-19T10:35:35.188 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:35.568 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 110/153 2023-10-19T10:35:35.595 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 110/153 2023-10-19T10:35:35.624 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5610.g6f3e6b48.e 111/153 2023-10-19T10:35:35.688 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5610.g6f3e6b48. 112/153 2023-10-19T10:35:35.718 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noa 113/153 2023-10-19T10:35:35.949 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noa 113/153 2023-10-19T10:35:35.994 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.n 114/153 2023-10-19T10:35:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.n 114/153 2023-10-19T10:35:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3 115/153 2023-10-19T10:35:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3 115/153 2023-10-19T10:35:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5610.g6f3e6b48.el 116/153 2023-10-19T10:35:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 117/153 2023-10-19T10:35:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 117/153 2023-10-19T10:35:37.362 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:37.363 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-19T10:35:37.363 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-19T10:35:37.363 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-19T10:35:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 118/153 2023-10-19T10:35:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 118/153 2023-10-19T10:35:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:18.0.0-5610.g6f3e6b48.el9.x86_64 119/153 2023-10-19T10:35:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 120/153 2023-10-19T10:35:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 120/153 2023-10-19T10:35:38.264 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:38.264 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-19T10:35:38.265 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-19T10:35:38.265 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-19T10:35:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6 121/153 2023-10-19T10:35:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6 121/153 2023-10-19T10:35:38.355 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:38.356 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-19T10:35:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 122/153 2023-10-19T10:35:38.422 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 122/153 2023-10-19T10:35:38.423 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:38.423 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-19T10:35:38.423 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-19T10:35:38.423 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-19T10:35:38.424 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:38.733 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 123/153 2023-10-19T10:35:38.757 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 123/153 2023-10-19T10:35:38.758 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-19T10:35:38.758 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-19T10:35:38.758 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-19T10:35:38.758 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-19T10:35:38.758 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:42.753 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:18.0.0-5610.g6f3e6b48.el9.x86_64 124/153 2023-10-19T10:35:42.803 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 125/153 2023-10-19T10:35:42.833 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 126/153 2023-10-19T10:35:43.027 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-19T10:35:43.145 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-19T10:35:43.196 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 129/153 2023-10-19T10:35:43.234 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-19T10:35:43.267 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-19T10:35:43.297 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_ 132/153 2023-10-19T10:35:43.327 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-19T10:35:43.410 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-19T10:35:43.491 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-19T10:35:43.525 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-19T10:35:44.162 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-19T10:35:44.199 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-19T10:35:44.220 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-19T10:35:44.270 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:18.0.0-5610.g6f3e6b48.el9.noarch 138/153 2023-10-19T10:35:45.187 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-19T10:35:45.281 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-19T10:35:45.323 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-19T10:35:45.415 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-19T10:35:45.455 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-19T10:35:45.470 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-19T10:35:45.470 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-19T10:35:45.537 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-19T10:35:45.561 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-19T10:35:45.589 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-19T10:35:45.612 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-19T10:35:45.618 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-19T10:35:45.619 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-19T10:35:45.814 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-19T10:35:45.815 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-19T10:35:45.906 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-19T10:35:45.931 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-19T10:35:45.931 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-19T10:35:46.015 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-19T10:35:46.083 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-19T10:35:50.485 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-19T10:35:50.485 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:18.0.0-5610.g6f3e6b48.el9.x86_64 1/153 2023-10-19T10:35:50.485 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2/153 2023-10-19T10:35:50.485 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 3/153 2023-10-19T10:35:50.486 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 4/153 2023-10-19T10:35:50.486 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6 5/153 2023-10-19T10:35:50.486 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 6/153 2023-10-19T10:35:50.486 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 7/153 2023-10-19T10:35:50.486 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 8/153 2023-10-19T10:35:50.487 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 9/153 2023-10-19T10:35:50.487 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 10/153 2023-10-19T10:35:50.487 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 11/153 2023-10-19T10:35:50.487 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:18.0.0-5610.g6f3e6b48.el9.x86_64 12/153 2023-10-19T10:35:50.487 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 13/153 2023-10-19T10:35:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_ 14/153 2023-10-19T10:35:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 15/153 2023-10-19T10:35:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 16/153 2023-10-19T10:35:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_6 17/153 2023-10-19T10:35:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86 18/153 2023-10-19T10:35:50.488 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 19/153 2023-10-19T10:35:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5610.g6f3e6b48.el 20/153 2023-10-19T10:35:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:18.0.0-5610.g6f3e6b48.el9. 21/153 2023-10-19T10:35:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:18.0.0-5610.g6f3e6b48.el9.x86_6 22/153 2023-10-19T10:35:50.489 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:18.0.0-5610.g6f3e6b48.el9.x86_64 23/153 2023-10-19T10:35:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 24/153 2023-10-19T10:35:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 25/153 2023-10-19T10:35:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 26/153 2023-10-19T10:35:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 27/153 2023-10-19T10:35:50.490 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 28/153 2023-10-19T10:35:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5610.g6f3e6b48. 29/153 2023-10-19T10:35:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noa 30/153 2023-10-19T10:35:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.n 31/153 2023-10-19T10:35:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3 32/153 2023-10-19T10:35:50.491 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5610.g6f3e6b48.el 33/153 2023-10-19T10:35:50.492 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 34/153 2023-10-19T10:35:50.492 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5610.g6f3e6b48.e 35/153 2023-10-19T10:35:50.492 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 36/153 2023-10-19T10:35:50.492 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 37/153 2023-10-19T10:35:50.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:18.0.0-5610.g6f3e6b48.el9.noarch 38/153 2023-10-19T10:35:50.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-19T10:35:50.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-19T10:35:50.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-19T10:35:50.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-19T10:35:50.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-19T10:35:50.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-19T10:35:50.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-19T10:35:50.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-19T10:35:50.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-19T10:35:50.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-19T10:35:50.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-19T10:35:50.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-19T10:35:50.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-19T10:35:50.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-19T10:35:50.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-19T10:35:50.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-19T10:35:50.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-19T10:35:50.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-19T10:35:50.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-19T10:35:50.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-19T10:35:50.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-19T10:35:50.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-19T10:35:50.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-19T10:35:50.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-19T10:35:50.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-19T10:35:50.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-19T10:35:50.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-19T10:35:50.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-19T10:35:50.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-19T10:35:50.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-19T10:35:50.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-19T10:35:50.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-19T10:35:50.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-19T10:35:50.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-19T10:35:50.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-19T10:35:50.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-19T10:35:50.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-19T10:35:50.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-19T10:35:50.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-19T10:35:50.512 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-19T10:35:50.512 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-19T10:35:50.512 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-19T10:35:50.513 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-19T10:35:50.513 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-19T10:35:50.513 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-19T10:35:50.513 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-19T10:35:50.514 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-19T10:35:50.514 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-19T10:35:50.514 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-19T10:35:50.514 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-19T10:35:50.515 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-19T10:35:50.515 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-19T10:35:50.515 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-19T10:35:50.515 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-19T10:35:50.515 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-19T10:35:50.516 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-19T10:35:50.516 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-19T10:35:50.516 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-19T10:35:50.516 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-19T10:35:50.516 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-19T10:35:50.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-19T10:35:50.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-19T10:35:50.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-19T10:35:50.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-19T10:35:50.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-19T10:35:50.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-19T10:35:50.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-19T10:35:50.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-19T10:35:50.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-19T10:35:50.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 108/153 2023-10-19T10:35:50.519 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-19T10:35:50.519 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-19T10:35:50.521 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-19T10:35:50.522 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-19T10:35:50.522 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-19T10:35:50.522 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-19T10:35:50.522 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-19T10:35:50.522 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-19T10:35:50.523 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-19T10:35:50.523 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-19T10:35:50.523 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-19T10:35:50.523 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-19T10:35:50.523 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-19T10:35:50.524 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-19T10:35:50.524 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-19T10:35:50.524 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-19T10:35:50.524 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-19T10:35:50.525 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-19T10:35:50.525 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-19T10:35:50.525 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-19T10:35:50.525 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-19T10:35:50.525 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-19T10:35:50.526 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-19T10:35:50.526 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-19T10:35:50.526 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-19T10:35:50.526 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 134/153 2023-10-19T10:35:50.527 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-19T10:35:50.527 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 136/153 2023-10-19T10:35:50.527 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-19T10:35:50.527 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-19T10:35:50.527 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-19T10:35:50.528 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-19T10:35:50.528 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-19T10:35:50.528 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-19T10:35:50.528 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-19T10:35:50.528 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-19T10:35:50.529 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-19T10:35:50.529 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-19T10:35:50.529 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-19T10:35:50.529 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-19T10:35:50.529 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-19T10:35:50.530 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-19T10:35:50.530 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-19T10:35:50.530 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-19T10:35:53.303 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-19T10:35:53.303 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:53.303 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2023-10-19T10:35:53.304 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:53.304 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:53.304 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:53.304 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-19T10:35:53.305 INFO:teuthology.orchestra.run.smithi050.stdout: acl-2.3.1-4.el9.x86_64 2023-10-19T10:35:53.305 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-19T10:35:53.305 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-19T10:35:53.305 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.306 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.306 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-19T10:35:53.306 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2023-10-19T10:35:53.306 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2023-10-19T10:35:53.307 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2023-10-19T10:35:53.307 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-19T10:35:53.307 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.307 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.308 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.308 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.308 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.308 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.308 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.309 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.309 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.309 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.309 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.310 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.310 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.311 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.311 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.311 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.311 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.312 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.312 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.312 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.312 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.313 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.313 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:18.0.0-5610.g6f3e6b48.el9.noarch 2023-10-19T10:35:53.313 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-19T10:35:53.313 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2023-10-19T10:35:53.313 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-19T10:35:53.314 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-19T10:35:53.314 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-19T10:35:53.314 INFO:teuthology.orchestra.run.smithi050.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-19T10:35:53.314 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-19T10:35:53.314 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-19T10:35:53.315 INFO:teuthology.orchestra.run.smithi050.stdout: jq-1.6-15.el9.x86_64 2023-10-19T10:35:53.315 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-19T10:35:53.315 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-19T10:35:53.315 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-19T10:35:53.315 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.316 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.316 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.316 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-19T10:35:53.316 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-19T10:35:53.316 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-19T10:35:53.316 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-19T10:35:53.317 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-19T10:35:53.317 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.317 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.317 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-19T10:35:53.317 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.318 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-19T10:35:53.318 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-19T10:35:53.318 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-19T10:35:53.318 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-19T10:35:53.318 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-19T10:35:53.318 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-19T10:35:53.319 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-19T10:35:53.319 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-19T10:35:53.319 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-19T10:35:53.319 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-19T10:35:53.319 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.11-1.el9.x86_64 2023-10-19T10:35:53.320 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-19T10:35:53.320 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-19T10:35:53.320 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-19T10:35:53.320 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-19T10:35:53.320 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-19T10:35:53.321 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-19T10:35:53.321 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-19T10:35:53.321 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.321 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.321 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.321 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-19T10:35:53.322 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-19T10:35:53.322 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-19T10:35:53.322 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-19T10:35:53.322 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-19T10:35:53.322 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-19T10:35:53.323 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-19T10:35:53.323 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-19T10:35:53.323 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-19T10:35:53.323 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-19T10:35:53.323 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-19T10:35:53.323 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-19T10:35:53.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-19T10:35:53.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-19T10:35:53.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-19T10:35:53.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-19T10:35:53.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-19T10:35:53.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-19T10:35:53.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-19T10:35:53.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-19T10:35:53.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-19T10:35:53.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-19T10:35:53.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-19T10:35:53.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-19T10:35:53.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-19T10:35:53.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-19T10:35:53.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-19T10:35:53.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2023-10-19T10:35:53.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-19T10:35:53.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-19T10:35:53.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-19T10:35:53.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-19T10:35:53.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-19T10:35:53.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-19T10:35:53.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-19T10:35:53.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-19T10:35:53.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-19T10:35:53.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-19T10:35:53.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-19T10:35:53.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-19T10:35:53.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-19T10:35:53.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-19T10:35:53.331 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-19T10:35:53.332 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-19T10:35:53.332 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-19T10:35:53.332 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-19T10:35:53.332 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-19T10:35:53.333 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-19T10:35:53.333 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-19T10:35:53.333 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.333 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.334 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:18.0.0-5610.g6f3e6b48.el9.x86_64 2023-10-19T10:35:53.334 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2023-10-19T10:35:53.334 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-19T10:35:53.334 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-19T10:35:53.334 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-19T10:35:53.335 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-19T10:35:53.335 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-19T10:35:53.336 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-19T10:35:53.336 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:53.336 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2023-10-19T10:35:53.733 DEBUG:teuthology.parallel:result is None 2023-10-19T10:35:53.733 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-19T10:35:53.955 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-19T10:35:53.978 INFO:teuthology.orchestra.run.smithi050.stdout:18.0.0-5610.g6f3e6b48.el9 2023-10-19T10:35:53.979 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5610.g6f3e6b48.el9 2023-10-19T10:35:53.979 INFO:teuthology.task.install:The correct ceph version 18.0.0-5610.g6f3e6b48 is installed. 2023-10-19T10:35:53.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-19T10:35:54.120 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-19T10:35:54.146 INFO:teuthology.orchestra.run.smithi183.stdout:18.0.0-5610.g6f3e6b48.el9 2023-10-19T10:35:54.147 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5610.g6f3e6b48.el9 2023-10-19T10:35:54.147 INFO:teuthology.task.install:The correct ceph version 18.0.0-5610.g6f3e6b48 is installed. 2023-10-19T10:35:54.147 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-19T10:35:54.148 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:54.148 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-19T10:35:54.184 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:54.184 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-19T10:35:54.219 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-19T10:35:54.219 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:54.220 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2023-10-19T10:35:54.257 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-19T10:35:54.330 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:54.330 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2023-10-19T10:35:54.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-19T10:35:54.431 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-19T10:35:54.431 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:54.432 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-19T10:35:54.464 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-19T10:35:54.537 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:54.537 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-19T10:35:54.567 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-19T10:35:54.635 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-19T10:35:54.636 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:54.636 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2023-10-19T10:35:54.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-19T10:35:54.741 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:54.742 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2023-10-19T10:35:54.769 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-19T10:35:54.837 INFO:teuthology.run_tasks:Running task ceph... 2023-10-19T10:35:55.076 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-19T10:35:55.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2023-10-19T10:35:55.080 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2023-10-19T10:35:55.110 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-19T10:35:55.110 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-19T10:35:55.153 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-19T10:35:55.182 INFO:tasks.ceph:Creating extra log directories... 2023-10-19T10:35:55.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-19T10:35:55.225 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-19T10:35:55.260 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-19T10:35:55.260 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-19T10:35:55.270 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-19T10:35:55.270 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:55.270 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-19T10:35:55.310 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-19T10:35:55.442 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-19T10:35:55.473 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:55.474 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-19T10:35:55.488 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-19T10:35:55.605 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-19T10:35:55.637 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-19T10:35:55.638 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps', 'file system flag refuse_client_session is set', 'Degraded data redundancy', 'MDS_CLIENTS_LAGGY', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-19T10:35:55.638 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271', 'branch': 'wip-63105', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}', 'email': None, 'first_in_suite': False, 'job_id': '7432271', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-63105'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps', 'file system flag refuse_client_session is set', 'Degraded data redundancy', 'MDS_CLIENTS_LAGGY', 'Reduced data availability'], 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-63105', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4476, 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-63105', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVcW0lHH7vdor8OZ0/odg/BpSOxSYa/C/m1DtWPz9HScDV548DEXi+MR4XATtCyy/F3z0te96Co8q/6rkp4Ffw=', 'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFTd5nvqZj0RT84qdMQXVb3bcX1g42IPAWhVYI7TaFUtS+MsLuT/FYYddm4d+bjWgOEDcH+V/K6pd1cofOY7wE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_recovery']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '202b180cb047e798fb131047314a862593f45403', 'timestamp': '2023-10-19_10:17:02', 'tube': 'smithi', 'user': 'dparmar', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-19T10:35:55.639 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-19T10:35:55.642 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-19T10:35:55.687 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2023-10-19T10:35:55.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2023-10-19T10:35:55.754 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:55.754 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2023-10-19T10:35:55.784 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-19T10:35:55.785 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2023-10-19T10:35:55.839 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-19T10:35:55.840 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:55.840 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-10-19T10:35:55.840 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:55.840 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:55.841 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-19 10:35:46.440703414 +0000 2023-10-19T10:35:55.841 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-19 10:33:37.839354547 +0000 2023-10-19T10:35:55.841 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-19 10:33:37.839354547 +0000 2023-10-19T10:35:55.841 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2023-10-19 10:33:37.839354547 +0000 2023-10-19T10:35:55.842 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-19T10:35:55.908 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-19T10:35:55.909 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-19T10:35:55.909 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000102879 s, 5.0 MB/s 2023-10-19T10:35:55.910 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-19T10:35:55.967 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2023-10-19T10:35:56.023 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-19T10:35:56.023 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:56.023 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-10-19T10:35:56.023 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:56.024 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:56.024 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-19 10:35:46.440703414 +0000 2023-10-19T10:35:56.024 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-19 10:33:38.339341964 +0000 2023-10-19T10:35:56.024 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-19 10:33:38.339341964 +0000 2023-10-19T10:35:56.024 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2023-10-19 10:33:38.339341964 +0000 2023-10-19T10:35:56.025 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-19T10:35:56.093 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-19T10:35:56.093 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-19T10:35:56.093 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000102918 s, 5.0 MB/s 2023-10-19T10:35:56.095 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-19T10:35:56.152 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2023-10-19T10:35:56.208 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-19T10:35:56.208 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:56.208 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 701 Links: 1 2023-10-19T10:35:56.208 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:56.209 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:56.209 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-19 10:35:46.441703389 +0000 2023-10-19T10:35:56.209 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-19 10:33:38.835329481 +0000 2023-10-19T10:35:56.209 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-19 10:33:38.835329481 +0000 2023-10-19T10:35:56.209 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2023-10-19 10:33:38.835329481 +0000 2023-10-19T10:35:56.210 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-19T10:35:56.277 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-19T10:35:56.278 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-19T10:35:56.278 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.4652e-05 s, 5.4 MB/s 2023-10-19T10:35:56.279 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-19T10:35:56.335 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2023-10-19T10:35:56.390 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-19T10:35:56.390 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:56.391 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 722 Links: 1 2023-10-19T10:35:56.391 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:56.391 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:56.391 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-10-19 10:35:46.441703389 +0000 2023-10-19T10:35:56.392 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-10-19 10:33:39.320317276 +0000 2023-10-19T10:35:56.392 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-10-19 10:33:39.320317276 +0000 2023-10-19T10:35:56.392 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2023-10-19 10:33:39.320317276 +0000 2023-10-19T10:35:56.392 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-19T10:35:56.459 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-10-19T10:35:56.460 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-10-19T10:35:56.460 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 7.659e-05 s, 6.7 MB/s 2023-10-19T10:35:56.461 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-19T10:35:56.517 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-19T10:35:56.518 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:56.518 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2023-10-19T10:35:56.535 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-19T10:35:56.536 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2023-10-19T10:35:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-19T10:35:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:56.592 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-10-19T10:35:56.592 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:56.592 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:56.592 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-19 10:35:26.008940483 +0000 2023-10-19T10:35:56.593 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-19 10:33:31.138185434 +0000 2023-10-19T10:35:56.593 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-19 10:33:31.138185434 +0000 2023-10-19T10:35:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2023-10-19 10:33:31.138185434 +0000 2023-10-19T10:35:56.593 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-19T10:35:56.658 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-19T10:35:56.659 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-19T10:35:56.659 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000185935 s, 2.8 MB/s 2023-10-19T10:35:56.660 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-19T10:35:56.717 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2023-10-19T10:35:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-19T10:35:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:56.774 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 681 Links: 1 2023-10-19T10:35:56.774 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:56.774 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:56.775 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-19 10:35:26.009940455 +0000 2023-10-19T10:35:56.775 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-19 10:33:31.629171915 +0000 2023-10-19T10:35:56.775 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-19 10:33:31.629171915 +0000 2023-10-19T10:35:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2023-10-19 10:33:31.629171915 +0000 2023-10-19T10:35:56.776 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-19T10:35:56.840 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-19T10:35:56.840 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-19T10:35:56.840 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000207032 s, 2.5 MB/s 2023-10-19T10:35:56.841 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-19T10:35:56.897 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2023-10-19T10:35:56.952 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-19T10:35:56.953 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:56.953 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 701 Links: 1 2023-10-19T10:35:56.953 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:56.953 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:56.953 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-19 10:35:26.009940455 +0000 2023-10-19T10:35:56.954 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-19 10:33:32.133158039 +0000 2023-10-19T10:35:56.954 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-19 10:33:32.133158039 +0000 2023-10-19T10:35:56.954 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2023-10-19 10:33:32.133158039 +0000 2023-10-19T10:35:56.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-19T10:35:57.018 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-19T10:35:57.018 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-19T10:35:57.018 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000200789 s, 2.5 MB/s 2023-10-19T10:35:57.019 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-19T10:35:57.075 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2023-10-19T10:35:57.129 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-19T10:35:57.129 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-19T10:35:57.130 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-10-19T10:35:57.130 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-19T10:35:57.130 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-10-19T10:35:57.130 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-19 10:35:26.010940427 +0000 2023-10-19T10:35:57.130 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-19 10:33:32.654143695 +0000 2023-10-19T10:35:57.131 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-19 10:33:32.654143695 +0000 2023-10-19T10:35:57.131 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2023-10-19 10:33:32.654143695 +0000 2023-10-19T10:35:57.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-19T10:35:57.194 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-19T10:35:57.194 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-19T10:35:57.195 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000175904 s, 2.9 MB/s 2023-10-19T10:35:57.196 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-19T10:35:57.251 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-19T10:35:57.251 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-19T10:35:57.252 INFO:tasks.ceph:Generating config... 2023-10-19T10:35:57.263 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-19T10:35:57.263 INFO:tasks.ceph:[client] debug client = 20 2023-10-19T10:35:57.264 INFO:tasks.ceph:[client] debug ms = 1 2023-10-19T10:35:57.264 INFO:tasks.ceph:[client] pid file = 2023-10-19T10:35:57.264 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-19T10:35:57.264 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-19T10:35:57.265 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-19T10:35:57.265 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-19T10:35:57.265 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-19T10:35:57.266 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-19T10:35:57.266 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-19T10:35:57.266 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-19T10:35:57.266 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-19T10:35:57.267 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-19T10:35:57.267 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-19T10:35:57.267 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-19T10:35:57.267 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-19T10:35:57.268 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-19T10:35:57.268 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-19T10:35:57.268 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-19T10:35:57.268 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-19T10:35:57.269 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-19T10:35:57.269 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-19T10:35:57.269 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-19T10:35:57.269 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-19T10:35:57.270 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-19T10:35:57.270 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-19T10:35:57.270 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-19T10:35:57.270 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-19T10:35:57.271 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-19T10:35:57.271 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-19T10:35:57.271 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-19T10:35:57.272 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-19T10:35:57.272 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-19T10:35:57.272 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-19T10:35:57.272 INFO:tasks.ceph:Setting up mon.a... 2023-10-19T10:35:57.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-19T10:35:57.330 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.keyring 2023-10-19T10:35:57.332 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-19T10:35:57.383 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-19T10:35:57.414 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.50'), ('mon.b', '172.21.15.183'), ('mon.c', '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]')] 2023-10-19T10:35:57.415 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.50,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-19T10:35:57.416 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:57.416 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-19T10:35:57.469 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.50 --add b 172.21.15.183 --addv c '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:35:57.555 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:35:57.555 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: generated fsid 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:35:57.555 INFO:teuthology.orchestra.run.smithi050.stdout:setting min_mon_release = pacific 2023-10-19T10:35:57.555 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 0 2023-10-19T10:35:57.556 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:35:57.556 INFO:teuthology.orchestra.run.smithi050.stdout:last_changed 2023-10-19T10:35:57.553609+0000 2023-10-19T10:35:57.556 INFO:teuthology.orchestra.run.smithi050.stdout:created 2023-10-19T10:35:57.553609+0000 2023-10-19T10:35:57.556 INFO:teuthology.orchestra.run.smithi050.stdout:min_mon_release 16 (pacific) 2023-10-19T10:35:57.556 INFO:teuthology.orchestra.run.smithi050.stdout:election_strategy: 1 2023-10-19T10:35:57.557 INFO:teuthology.orchestra.run.smithi050.stdout:0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2023-10-19T10:35:57.557 INFO:teuthology.orchestra.run.smithi050.stdout:1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2023-10-19T10:35:57.557 INFO:teuthology.orchestra.run.smithi050.stdout:2: [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] mon.c 2023-10-19T10:35:57.557 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-19T10:35:57.558 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-19T10:35:57.574 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2b47c9ed-f967-49ff-9d46-d95ee8c39a83... 2023-10-19T10:35:57.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-19T10:35:57.617 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-19T10:35:57.666 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2023-10-19T10:35:57.667 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2023-10-19T10:35:57.667 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-19T10:35:57.667 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2023-10-19T10:35:57.667 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.668 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2023-10-19T10:35:57.668 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.668 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2023-10-19T10:35:57.668 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.668 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2023-10-19T10:35:57.669 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2023-10-19T10:35:57.669 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.669 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2023-10-19T10:35:57.669 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2023-10-19T10:35:57.669 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.670 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-19T10:35:57.670 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2023-10-19T10:35:57.670 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.670 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2023-10-19T10:35:57.670 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-19T10:35:57.671 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.671 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2023-10-19T10:35:57.671 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.671 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2023-10-19T10:35:57.671 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2023-10-19T10:35:57.671 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2023-10-19T10:35:57.672 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2023-10-19T10:35:57.672 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2023-10-19T10:35:57.672 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2023-10-19T10:35:57.672 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2023-10-19T10:35:57.672 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-19T10:35:57.673 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-19T10:35:57.673 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2023-10-19T10:35:57.673 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.673 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-19T10:35:57.673 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.674 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2023-10-19T10:35:57.674 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.674 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2023-10-19T10:35:57.674 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.674 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2023-10-19T10:35:57.674 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2023-10-19T10:35:57.675 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2023-10-19T10:35:57.675 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = "172.21.15.50,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2023-10-19T10:35:57.675 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:35:57.675 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.675 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2023-10-19T10:35:57.676 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2023-10-19T10:35:57.676 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.676 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2023-10-19T10:35:57.676 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2023-10-19T10:35:57.676 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2023-10-19T10:35:57.677 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.677 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2023-10-19T10:35:57.677 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2023-10-19T10:35:57.677 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.677 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2023-10-19T10:35:57.678 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2023-10-19T10:35:57.678 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2023-10-19T10:35:57.678 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.678 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2023-10-19T10:35:57.678 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2023-10-19T10:35:57.679 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.679 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2023-10-19T10:35:57.679 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.679 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2023-10-19T10:35:57.679 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.680 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2023-10-19T10:35:57.680 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.680 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2023-10-19T10:35:57.680 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2023-10-19T10:35:57.680 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.680 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2023-10-19T10:35:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2023-10-19T10:35:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2023-10-19T10:35:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2023-10-19T10:35:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 20 2023-10-19T10:35:57.681 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 20 2023-10-19T10:35:57.682 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-19T10:35:57.682 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2023-10-19T10:35:57.682 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 10 2023-10-19T10:35:57.682 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2023-10-19T10:35:57.682 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2023-10-19T10:35:57.682 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2023-10-19T10:35:57.683 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2023-10-19T10:35:57.683 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2023-10-19T10:35:57.683 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2023-10-19T10:35:57.683 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.683 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2023-10-19T10:35:57.684 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-19T10:35:57.684 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2023-10-19T10:35:57.684 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2023-10-19T10:35:57.684 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2023-10-19T10:35:57.684 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2023-10-19T10:35:57.684 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2023-10-19T10:35:57.685 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2023-10-19T10:35:57.685 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.685 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2023-10-19T10:35:57.685 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-19T10:35:57.685 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2023-10-19T10:35:57.686 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2023-10-19T10:35:57.686 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2023-10-19T10:35:57.686 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2023-10-19T10:35:57.686 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2023-10-19T10:35:57.687 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2023-10-19T10:35:57.687 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2023-10-19T10:35:57.687 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2023-10-19T10:35:57.688 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2023-10-19T10:35:57.688 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.688 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-19T10:35:57.688 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2023-10-19T10:35:57.688 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2023-10-19T10:35:57.689 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.689 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2023-10-19T10:35:57.689 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-19T10:35:57.689 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-19T10:35:57.689 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.690 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2023-10-19T10:35:57.690 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2023-10-19T10:35:57.690 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.690 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2023-10-19T10:35:57.690 INFO:teuthology.orchestra.run.smithi183.stdout: mon op complaint time = 120 2023-10-19T10:35:57.690 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-19T10:35:57.691 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2023-10-19T10:35:57.691 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2023-10-19T10:35:57.691 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2023-10-19T10:35:57.691 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2023-10-19T10:35:57.691 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-19T10:35:57.692 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-19T10:35:57.692 INFO:teuthology.orchestra.run.smithi183.stdout: client mount timeout = 600 2023-10-19T10:35:57.692 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2023-10-19T10:35:57.692 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-19T10:35:57.692 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = "" 2023-10-19T10:35:57.692 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2023-10-19T10:35:57.693 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2023-10-19T10:35:57.693 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2023-10-19T10:35:57.693 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.b] 2023-10-19T10:35:57.693 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.c] 2023-10-19T10:35:57.693 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2023-10-19T10:35:57.694 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2023-10-19T10:35:57.694 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2023-10-19T10:35:57.694 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2023-10-19T10:35:57.694 INFO:teuthology.orchestra.run.smithi183.stdout:[mds] 2023-10-19T10:35:57.694 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds = 20 2023-10-19T10:35:57.695 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds balancer = 20 2023-10-19T10:35:57.695 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-19T10:35:57.695 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug frag = True 2023-10-19T10:35:57.695 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug scatterstat = True 2023-10-19T10:35:57.695 INFO:teuthology.orchestra.run.smithi183.stdout: mds op complaint time = 180 2023-10-19T10:35:57.695 INFO:teuthology.orchestra.run.smithi183.stdout: mds verify scatter = True 2023-10-19T10:35:57.696 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2023-10-19T10:35:57.696 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2023-10-19T10:35:57.696 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2023-10-19T10:35:57.698 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2023-10-19T10:35:57.698 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2023-10-19T10:35:57.698 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-19T10:35:57.698 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2023-10-19T10:35:57.698 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.699 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2023-10-19T10:35:57.699 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.699 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2023-10-19T10:35:57.699 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.699 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2023-10-19T10:35:57.700 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2023-10-19T10:35:57.700 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.700 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2023-10-19T10:35:57.700 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2023-10-19T10:35:57.701 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.701 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-19T10:35:57.701 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2023-10-19T10:35:57.701 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.701 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2023-10-19T10:35:57.701 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-19T10:35:57.702 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.702 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2023-10-19T10:35:57.702 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.706 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2023-10-19T10:35:57.707 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2023-10-19T10:35:57.707 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2023-10-19T10:35:57.707 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2023-10-19T10:35:57.707 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2023-10-19T10:35:57.707 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2023-10-19T10:35:57.708 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2023-10-19T10:35:57.708 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-19T10:35:57.708 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-19T10:35:57.708 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2023-10-19T10:35:57.708 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.709 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-19T10:35:57.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.709 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2023-10-19T10:35:57.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.709 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2023-10-19T10:35:57.710 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.710 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2023-10-19T10:35:57.710 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2023-10-19T10:35:57.710 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2023-10-19T10:35:57.710 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = "172.21.15.50,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2023-10-19T10:35:57.711 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:35:57.711 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.711 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2023-10-19T10:35:57.711 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2023-10-19T10:35:57.711 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.711 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2023-10-19T10:35:57.712 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2023-10-19T10:35:57.712 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2023-10-19T10:35:57.712 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.712 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2023-10-19T10:35:57.712 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2023-10-19T10:35:57.713 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.713 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2023-10-19T10:35:57.713 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2023-10-19T10:35:57.713 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2023-10-19T10:35:57.713 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.713 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2023-10-19T10:35:57.714 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2023-10-19T10:35:57.714 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.714 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2023-10-19T10:35:57.714 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.714 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2023-10-19T10:35:57.715 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.715 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2023-10-19T10:35:57.715 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.715 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2023-10-19T10:35:57.715 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2023-10-19T10:35:57.716 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.716 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2023-10-19T10:35:57.716 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2023-10-19T10:35:57.716 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2023-10-19T10:35:57.716 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2023-10-19T10:35:57.716 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 20 2023-10-19T10:35:57.717 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 20 2023-10-19T10:35:57.717 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-19T10:35:57.717 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2023-10-19T10:35:57.717 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 10 2023-10-19T10:35:57.717 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2023-10-19T10:35:57.718 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2023-10-19T10:35:57.718 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2023-10-19T10:35:57.718 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2023-10-19T10:35:57.718 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2023-10-19T10:35:57.718 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2023-10-19T10:35:57.718 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.719 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2023-10-19T10:35:57.719 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-19T10:35:57.719 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2023-10-19T10:35:57.719 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-10-19T10:35:57.719 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-10-19T10:35:57.720 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-10-19T10:35:57.720 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-10-19T10:35:57.720 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2023-10-19T10:35:57.720 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.720 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2023-10-19T10:35:57.720 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-19T10:35:57.721 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-10-19T10:35:57.721 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2023-10-19T10:35:57.721 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-10-19T10:35:57.721 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2023-10-19T10:35:57.721 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2023-10-19T10:35:57.722 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-10-19T10:35:57.722 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2023-10-19T10:35:57.722 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2023-10-19T10:35:57.722 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-10-19T10:35:57.722 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.722 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-19T10:35:57.723 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2023-10-19T10:35:57.723 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2023-10-19T10:35:57.723 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.723 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2023-10-19T10:35:57.723 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-19T10:35:57.724 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-19T10:35:57.724 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.724 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2023-10-19T10:35:57.724 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2023-10-19T10:35:57.724 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.725 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2023-10-19T10:35:57.725 INFO:teuthology.orchestra.run.smithi050.stdout: mon op complaint time = 120 2023-10-19T10:35:57.725 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:35:57.725 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2023-10-19T10:35:57.725 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2023-10-19T10:35:57.725 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2023-10-19T10:35:57.726 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2023-10-19T10:35:57.726 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-19T10:35:57.726 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-19T10:35:57.726 INFO:teuthology.orchestra.run.smithi050.stdout: client mount timeout = 600 2023-10-19T10:35:57.726 INFO:teuthology.orchestra.run.smithi050.stdout: debug client = 20 2023-10-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = "" 2023-10-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2023-10-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2023-10-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2023-10-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.b] 2023-10-19T10:35:57.728 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.c] 2023-10-19T10:35:57.728 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2023-10-19T10:35:57.728 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2023-10-19T10:35:57.728 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2023-10-19T10:35:57.728 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2023-10-19T10:35:57.729 INFO:teuthology.orchestra.run.smithi050.stdout:[mds] 2023-10-19T10:35:57.729 INFO:teuthology.orchestra.run.smithi050.stdout: debug mds = 20 2023-10-19T10:35:57.729 INFO:teuthology.orchestra.run.smithi050.stdout: debug mds balancer = 20 2023-10-19T10:35:57.729 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-10-19T10:35:57.729 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug frag = True 2023-10-19T10:35:57.729 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug scatterstat = True 2023-10-19T10:35:57.730 INFO:teuthology.orchestra.run.smithi050.stdout: mds op complaint time = 180 2023-10-19T10:35:57.730 INFO:teuthology.orchestra.run.smithi050.stdout: mds verify scatter = True 2023-10-19T10:35:57.730 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2023-10-19T10:35:57.730 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2023-10-19T10:35:57.730 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2023-10-19T10:35:57.731 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-19T10:35:57.731 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-19T10:35:57.791 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-19T10:35:57.791 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:57.792 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-19T10:35:57.806 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:57.807 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-19T10:35:57.861 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:35:57.861 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:57.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-19T10:35:57.862 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-19T10:35:57.947 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:35:57.947 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:35:58.002 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:35:58.003 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:58.003 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-19T10:35:58.003 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-19T10:35:58.042 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:35:58.043 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:35:58.097 INFO:tasks.ceph:Setting up mon nodes... 2023-10-19T10:35:58.098 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-19T10:35:58.098 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-19T10:35:58.160 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-19T10:35:58.163 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-19T10:35:58.231 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-19T10:35:58.233 INFO:tasks.ceph:Setting up mds nodes... 2023-10-19T10:35:58.234 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-19T10:35:58.295 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-19T10:35:58.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-19T10:35:58.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-19T10:35:58.429 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-19T10:35:58.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-19T10:35:58.462 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-19T10:35:58.516 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-19T10:35:58.519 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-19T10:35:58.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-19T10:35:58.640 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-19T10:35:58.643 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-19T10:35:58.670 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-19T10:35:58.670 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-19T10:35:58.717 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-19T10:35:58.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-19T10:35:58.782 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-19T10:35:58.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-19T10:35:58.842 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-19T10:35:58.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-19T10:35:58.941 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-19T10:35:58.954 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-19T10:35:58.954 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-19T10:35:58.955 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-19T10:35:58.986 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-19T10:35:58.986 INFO:tasks.ceph:role: osd.0 2023-10-19T10:35:58.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:35:58.987 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-19T10:35:59.056 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:35:59.056 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:35:59.057 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:35:59.057 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:35:59.057 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:35:59.057 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:35:59.057 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:35:59.058 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:35:59.058 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:35:59.058 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:35:59.089 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2023-10-19T10:35:59.091 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-19T10:35:59.091 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-19T10:35:59.166 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-19T10:35:59.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-19T10:35:59.312 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-19T10:35:59.312 INFO:tasks.ceph:role: osd.1 2023-10-19T10:35:59.313 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:35:59.313 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-19T10:35:59.382 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:35:59.383 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:35:59.383 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:35:59.383 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:35:59.383 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:35:59.384 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:35:59.384 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:35:59.384 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:35:59.384 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:35:59.384 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:35:59.414 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2023-10-19T10:35:59.416 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-19T10:35:59.416 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-19T10:35:59.492 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-19T10:35:59.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-19T10:35:59.637 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-19T10:35:59.637 INFO:tasks.ceph:role: osd.2 2023-10-19T10:35:59.637 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:35:59.638 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-19T10:35:59.706 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:35:59.707 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:35:59.707 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:35:59.707 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:35:59.707 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:35:59.708 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:35:59.708 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:35:59.708 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:35:59.708 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:35:59.708 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:35:59.742 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2023-10-19T10:35:59.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-19T10:35:59.743 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-19T10:35:59.817 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-19T10:35:59.891 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-19T10:35:59.962 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-19T10:35:59.962 INFO:tasks.ceph:role: osd.3 2023-10-19T10:35:59.963 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:35:59.963 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-19T10:36:00.032 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:36:00.033 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:36:00.033 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:36:00.033 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:36:00.033 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:36:00.034 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:36:00.034 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:36:00.034 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:36:00.034 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:36:00.034 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:36:00.065 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2023-10-19T10:36:00.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-10-19T10:36:00.068 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-19T10:36:00.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-19T10:36:00.215 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:00.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:00.328+0000 7f5c297be500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-19T10:36:00.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:00.328+0000 7f5c297be500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-19T10:36:00.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:00.328+0000 7f5c297be500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-19T10:36:00.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:00.328+0000 7f5c297be500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-19T10:36:00.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:00.328+0000 7f5c297be500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-19T10:36:00.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:00.329+0000 7f5c297be500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-19T10:36:03.180 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-19T10:36:03.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:03.308+0000 7faf42725500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-19T10:36:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:03.309+0000 7faf42725500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-19T10:36:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:03.309+0000 7faf42725500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-19T10:36:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:03.309+0000 7faf42725500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-19T10:36:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:03.309+0000 7faf42725500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-19T10:36:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:03.309+0000 7faf42725500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-19T10:36:06.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-19T10:36:06.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:06.302+0000 7f0b1acc9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-19T10:36:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:06.302+0000 7f0b1acc9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-19T10:36:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:06.302+0000 7f0b1acc9500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-19T10:36:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:06.302+0000 7f0b1acc9500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-19T10:36:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:06.302+0000 7f0b1acc9500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-19T10:36:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:06.303+0000 7f0b1acc9500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-19T10:36:09.149 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-19T10:36:09.185 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:09.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:09.281+0000 7f466d216500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-19T10:36:09.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:09.281+0000 7f466d216500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-19T10:36:09.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:09.281+0000 7f466d216500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-19T10:36:09.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:09.281+0000 7f466d216500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-19T10:36:09.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:09.281+0000 7f466d216500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-19T10:36:09.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:09.282+0000 7f466d216500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-19T10:36:12.135 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-19T10:36:12.170 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-19T10:36:12.203 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-19T10:36:12.203 INFO:tasks.ceph:role: osd.4 2023-10-19T10:36:12.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:36:12.204 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-19T10:36:12.281 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:36:12.281 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:36:12.281 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:36:12.281 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:36:12.282 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:36:12.282 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:36:12.282 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:36:12.282 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:36:12.282 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:36:12.283 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:36:12.781 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-19T10:36:12.784 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-19T10:36:12.784 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-19T10:36:12.909 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-19T10:36:12.945 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-19T10:36:13.014 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-19T10:36:13.015 INFO:tasks.ceph:role: osd.5 2023-10-19T10:36:13.015 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:36:13.015 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-19T10:36:13.091 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:36:13.092 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:36:13.092 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:36:13.092 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:36:13.093 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:36:13.093 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:36:13.094 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:36:13.094 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:36:13.095 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:36:13.095 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:36:13.558 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-19T10:36:13.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-19T10:36:13.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-19T10:36:13.664 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-19T10:36:13.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-19T10:36:13.766 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-19T10:36:13.767 INFO:tasks.ceph:role: osd.6 2023-10-19T10:36:13.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:36:13.767 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-19T10:36:13.842 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:36:13.843 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:36:13.843 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:36:13.843 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:36:13.844 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:36:13.844 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:36:13.844 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:36:13.844 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:36:13.844 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:36:13.845 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:36:14.346 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-19T10:36:14.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-19T10:36:14.348 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-19T10:36:14.506 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-19T10:36:14.540 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-19T10:36:14.610 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-19T10:36:14.610 INFO:tasks.ceph:role: osd.7 2023-10-19T10:36:14.610 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:36:14.611 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-19T10:36:14.686 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-19T10:36:14.686 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-19T10:36:14.686 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-19T10:36:14.686 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-19T10:36:14.687 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-19T10:36:14.687 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-19T10:36:14.687 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-19T10:36:14.687 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-19T10:36:14.687 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-19T10:36:14.688 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-19T10:36:15.188 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-19T10:36:15.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-19T10:36:15.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-19T10:36:15.295 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-19T10:36:15.333 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:15.434 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:15.432+0000 7f633f425500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-19T10:36:15.435 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:15.432+0000 7f633f425500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-19T10:36:15.435 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:15.432+0000 7f633f425500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-19T10:36:15.435 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:15.432+0000 7f633f425500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-19T10:36:15.435 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:15.432+0000 7f633f425500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-19T10:36:15.436 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:15.432+0000 7f633f425500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-19T10:36:18.283 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-19T10:36:18.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:18.406 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:18.402+0000 7f089b61c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-19T10:36:18.406 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:18.402+0000 7f089b61c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-19T10:36:18.407 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:18.402+0000 7f089b61c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-19T10:36:18.407 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:18.402+0000 7f089b61c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-19T10:36:18.407 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:18.402+0000 7f089b61c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-19T10:36:18.407 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:18.403+0000 7f089b61c500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-19T10:36:21.253 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-19T10:36:21.282 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:21.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:21.367+0000 7f9af82b6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-19T10:36:21.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:21.368+0000 7f9af82b6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-19T10:36:21.371 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:21.368+0000 7f9af82b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-19T10:36:21.371 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:21.368+0000 7f9af82b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-19T10:36:21.371 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:21.368+0000 7f9af82b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-19T10:36:21.371 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:21.368+0000 7f9af82b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-19T10:36:24.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-19T10:36:24.252 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:24.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:24.340+0000 7f17c69ad500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-19T10:36:24.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:24.340+0000 7f17c69ad500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-19T10:36:24.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:24.340+0000 7f17c69ad500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-19T10:36:24.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:24.341+0000 7f17c69ad500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-19T10:36:24.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:24.341+0000 7f17c69ad500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-19T10:36:24.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:24.341+0000 7f17c69ad500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-19T10:36:25.640 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:36:25.644 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:36:27.204 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-19T10:36:27.234 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-19T10:36:27.234 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.235 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-19T10:36:27.271 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.272 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-19T10:36:27.346 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.346 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-19T10:36:27.420 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.420 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-19T10:36:27.493 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-19T10:36:27.566 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-19T10:36:27.637 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:27.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-19T10:36:27.709 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:27.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-19T10:36:27.742 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:27.742 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-19T10:36:27.812 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:27.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-19T10:36:27.881 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:27.882 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-19T10:36:27.951 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:27.951 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-19T10:36:28.020 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:28.020 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-19T10:36:28.088 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:28.089 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-19T10:36:28.155 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:28.155 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-19T10:36:28.172 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:28.173 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-19T10:36:28.229 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:28.229 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-19T10:36:28.245 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:28.245 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-19T10:36:28.300 INFO:tasks.ceph:Adding keys to all mons... 2023-10-19T10:36:28.301 DEBUG:teuthology.orchestra.run.smithi050:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-19T10:36:28.303 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-19T10:36:28.344 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.y] 2023-10-19T10:36:28.344 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlv7p8CRAAg8SiGH5E57/O6+UlWpgoCw== 2023-10-19T10:36:28.344 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2023-10-19T10:36:28.344 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlRrqKERAA4Ap/UVvrMlJVVcCkt0gd6Q== 2023-10-19T10:36:28.345 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2023-10-19T10:36:28.345 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlg+qDGRAAfdK7WLKio8hmPYJkWib0Nw== 2023-10-19T10:36:28.345 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2023-10-19T10:36:28.345 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCQBjFlvGmmExAAC5CLzIPkJo/EBVkpnX4ECQ== 2023-10-19T10:36:28.345 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2023-10-19T10:36:28.346 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCTBjFlIkp4EhAACncwsFhJPpqc8svxntHDLg== 2023-10-19T10:36:28.346 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2023-10-19T10:36:28.346 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCWBjFl06cYEhAA/rxMv0AQfiRoZJkquKcaTQ== 2023-10-19T10:36:28.346 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.3] 2023-10-19T10:36:28.347 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCZBjFl0/TUEBAAWoTV+/BBZVx4BIa5Bz57aw== 2023-10-19T10:36:28.347 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.x] 2023-10-19T10:36:28.348 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlXJGxDRAAGXHVR7kbvvZrRnoXOw18kg== 2023-10-19T10:36:28.348 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2023-10-19T10:36:28.348 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlVUq6HhAAxkq93v09PLVHLcq0RgMvYA== 2023-10-19T10:36:28.349 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2023-10-19T10:36:28.349 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlxk4cJhAAAf3p7z1wG6pB0TYpyKoEmg== 2023-10-19T10:36:28.349 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.4] 2023-10-19T10:36:28.350 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCfBjFlA3/OGRAA3aeeufXss+eAcAxoT8scOw== 2023-10-19T10:36:28.350 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.5] 2023-10-19T10:36:28.351 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCiBjFl5y8KGBAADp0V5u40szv14/1Jhlco/A== 2023-10-19T10:36:28.351 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.6] 2023-10-19T10:36:28.351 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQClBjFljPz4FRAAQfXMoK9aLXBsjX60h+cNDQ== 2023-10-19T10:36:28.352 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.7] 2023-10-19T10:36:28.352 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCoBjFljvFbFBAAOUga0rBE6sm1IHvuQzi7Ww== 2023-10-19T10:36:28.352 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2023-10-19T10:36:28.353 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlR3SvKhAAQGRIzy2kDJzeLJAJbvCD9Q== 2023-10-19T10:36:28.353 INFO:teuthology.orchestra.run.smithi050.stdout:[client.2] 2023-10-19T10:36:28.353 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlJyiOLhAAosLYQg1Va20wfFzst/qkAA== 2023-10-19T10:36:28.354 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2023-10-19T10:36:28.354 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlHmwhMhAAxbIws9tQaedL+MvOUYfljA== 2023-10-19T10:36:28.354 INFO:teuthology.orchestra.run.smithi050.stdout:[client.3] 2023-10-19T10:36:28.355 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlgb7iNxAAIkDDvbzY35oqbjo2R6duaA== 2023-10-19T10:36:28.366 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.y] 2023-10-19T10:36:28.367 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlv7p8CRAAg8SiGH5E57/O6+UlWpgoCw== 2023-10-19T10:36:28.367 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2023-10-19T10:36:28.367 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlRrqKERAA4Ap/UVvrMlJVVcCkt0gd6Q== 2023-10-19T10:36:28.368 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2023-10-19T10:36:28.368 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlg+qDGRAAfdK7WLKio8hmPYJkWib0Nw== 2023-10-19T10:36:28.368 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2023-10-19T10:36:28.368 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCQBjFlvGmmExAAC5CLzIPkJo/EBVkpnX4ECQ== 2023-10-19T10:36:28.368 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2023-10-19T10:36:28.369 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCTBjFlIkp4EhAACncwsFhJPpqc8svxntHDLg== 2023-10-19T10:36:28.369 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2023-10-19T10:36:28.369 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCWBjFl06cYEhAA/rxMv0AQfiRoZJkquKcaTQ== 2023-10-19T10:36:28.369 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.3] 2023-10-19T10:36:28.370 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCZBjFl0/TUEBAAWoTV+/BBZVx4BIa5Bz57aw== 2023-10-19T10:36:28.370 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2023-10-19T10:36:28.370 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlXJGxDRAAGXHVR7kbvvZrRnoXOw18kg== 2023-10-19T10:36:28.370 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2023-10-19T10:36:28.370 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlVUq6HhAAxkq93v09PLVHLcq0RgMvYA== 2023-10-19T10:36:28.371 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2023-10-19T10:36:28.371 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlxk4cJhAAAf3p7z1wG6pB0TYpyKoEmg== 2023-10-19T10:36:28.371 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.4] 2023-10-19T10:36:28.371 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCfBjFlA3/OGRAA3aeeufXss+eAcAxoT8scOw== 2023-10-19T10:36:28.371 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.5] 2023-10-19T10:36:28.372 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCiBjFl5y8KGBAADp0V5u40szv14/1Jhlco/A== 2023-10-19T10:36:28.372 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.6] 2023-10-19T10:36:28.372 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQClBjFljPz4FRAAQfXMoK9aLXBsjX60h+cNDQ== 2023-10-19T10:36:28.372 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.7] 2023-10-19T10:36:28.373 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCoBjFljvFbFBAAOUga0rBE6sm1IHvuQzi7Ww== 2023-10-19T10:36:28.373 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2023-10-19T10:36:28.373 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlR3SvKhAAQGRIzy2kDJzeLJAJbvCD9Q== 2023-10-19T10:36:28.373 INFO:teuthology.orchestra.run.smithi183.stdout:[client.2] 2023-10-19T10:36:28.374 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlJyiOLhAAosLYQg1Va20wfFzst/qkAA== 2023-10-19T10:36:28.374 INFO:teuthology.orchestra.run.smithi183.stdout:[client.1] 2023-10-19T10:36:28.374 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlHmwhMhAAxbIws9tQaedL+MvOUYfljA== 2023-10-19T10:36:28.374 INFO:teuthology.orchestra.run.smithi183.stdout:[client.3] 2023-10-19T10:36:28.375 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCOBjFlgb7iNxAAIkDDvbzY35oqbjo2R6duaA== 2023-10-19T10:36:28.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-19T10:36:28.399 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-19T10:36:28.469 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.471 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.529 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.532 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.589 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.592 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.645 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.648 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.701 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.703 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.759 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-19T10:36:28.814 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-19T10:36:28.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.870 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.928 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-19T10:36:28.984 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:28.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.039 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.095 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.151 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.154 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-19T10:36:29.207 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.267 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.323 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.382 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.384 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-19T10:36:29.440 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-19T10:36:29.441 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-19T10:36:29.472 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-19T10:36:29.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-19T10:36:29.603 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-19T10:36:29.630 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-19T10:36:29.726 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-19T10:36:29.753 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-19T10:36:29.820 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-19T10:36:29.918 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-19T10:36:29.945 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:29.948 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-19T10:36:29.999 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-19T10:36:30.000 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-19T10:36:30.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-19T10:36:30.003 INFO:tasks.ceph.mon.a:Started 2023-10-19T10:36:30.004 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-19T10:36:30.004 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-19T10:36:30.042 INFO:tasks.ceph.mon.b:Started 2023-10-19T10:36:30.042 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-19T10:36:30.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-19T10:36:30.045 INFO:tasks.ceph.mon.c:Started 2023-10-19T10:36:30.045 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-19T10:36:30.045 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-19T10:36:30.046 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-19T10:36:30.048 INFO:tasks.ceph.mgr.y:Started 2023-10-19T10:36:30.049 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-19T10:36:30.049 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-19T10:36:30.051 INFO:tasks.ceph.mgr.x:Started 2023-10-19T10:36:30.051 INFO:tasks.ceph:Setting crush tunables to default 2023-10-19T10:36:30.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd crush tunables default 2023-10-19T10:36:30.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.157+0000 7f56b5fa7640 1 Processor -- start 2023-10-19T10:36:30.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.157+0000 7f56b5fa7640 1 -- start start 2023-10-19T10:36:30.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56b5fa7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 0x7f56b0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56b5fa7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b00d0070 con 0x7f56b015fbc0 2023-10-19T10:36:30.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56b5fa7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b00d02c0 con 0x7f56b00b2400 2023-10-19T10:36:30.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56b5fa7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b0160fe0 con 0x7f56b00cfc00 2023-10-19T10:36:30.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 0x7f56b0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 0x7f56b0160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46626/0 (socket says 172.21.15.50:46626) 2023-10-19T10:36:30.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 learned_addr learned my addr 172.21.15.50:0/4170956875 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:30.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 msgr2=0x7f56b0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-19T10:36:30.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 msgr2=0x7f56b0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-19T10:36:30.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 --2- 172.21.15.50:0/4170956875 >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 0x7f56b0160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-19T10:36:30.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.158+0000 7f56affff640 1 --2- 172.21.15.50:0/4170956875 >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 0x7f56b0160aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-19T10:36:30.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.352+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134264509 0 0) 0x7f56b00d0070 con 0x7f56b015fbc0 2023-10-19T10:36:30.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.352+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5698003520 con 0x7f56b015fbc0 2023-10-19T10:36:30.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623402609 0 0) 0x7f5698003520 con 0x7f56b015fbc0 2023-10-19T10:36:30.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 msgr2=0x7f56b0160aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-19T10:36:30.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 --2- 172.21.15.50:0/4170956875 >> v2:172.21.15.50:3300/0 conn(0x7f56b00cfc00 0x7f56b0160aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 >> v1:172.21.15.183:6789/0 conn(0x7f56b00b2400 legacy=0x7f56b015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:30.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b00d05a0 con 0x7f56b015fbc0 2023-10-19T10:36:30.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f56a0001670 con 0x7f56b015fbc0 2023-10-19T10:36:30.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 >> v1:172.21.15.50:6789/0 conn(0x7f56b015fbc0 legacy=0x7f56b00b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:30.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698003970 0x7f5698003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56980047c0 0x7f5698004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698005180 0x7f569800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0160fe0 con 0x7f5698005180 2023-10-19T10:36:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569800c0d0 con 0x7f5698003970 2023-10-19T10:36:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569800c290 con 0x7f56980047c0 2023-10-19T10:36:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56af7fe640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698003970 0x7f5698003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.353+0000 7f56aeffd640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56980047c0 0x7f5698004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56affff640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698005180 0x7f569800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56980047c0 msgr2=0x7f5698004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:30.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56affff640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56980047c0 0x7f5698004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698003970 msgr2=0x7f5698003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:30.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56affff640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698003970 0x7f5698003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5698003520 con 0x7f5698005180 2023-10-19T10:36:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.354+0000 7f56aeffd640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56980047c0 0x7f5698004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56affff640 1 -- 172.21.15.50:0/4170956875 reap_dead start 2023-10-19T10:36:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56affff640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698005180 0x7f569800b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009cc0 tx=0x7f56980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03702e296637eea server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a802cb10 con 0x7f5698005180 2023-10-19T10:36:30.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8030070 con 0x7f5698005180 2023-10-19T10:36:30.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56ae7fc640 1 -- 172.21.15.50:0/4170956875 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8036410 con 0x7f5698005180 2023-10-19T10:36:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56b5fa7640 1 -- 172.21.15.50:0/4170956875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698005180 msgr2=0x7f569800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.355+0000 7f56b5fa7640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698005180 0x7f569800b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009cc0 tx=0x7f56980057f0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.356+0000 7f56b5fa7640 1 -- 172.21.15.50:0/4170956875 shutdown_connections 2023-10-19T10:36:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.356+0000 7f56b5fa7640 1 --2- 172.21.15.50:0/4170956875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698005180 0x7f569800b980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.356+0000 7f56b5fa7640 1 -- 172.21.15.50:0/4170956875 >> 172.21.15.50:0/4170956875 conn(0x7f56b006f3f0 msgr2=0x7f56b006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.356+0000 7f56b5fa7640 1 -- 172.21.15.50:0/4170956875 shutdown_connections 2023-10-19T10:36:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.356+0000 7f56b5fa7640 1 -- 172.21.15.50:0/4170956875 wait complete. 2023-10-19T10:36:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.357+0000 7f56b5fa7640 1 Processor -- start 2023-10-19T10:36:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.357+0000 7f56b5fa7640 1 -- start start 2023-10-19T10:36:30.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56b5fa7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698005180 0x7f56b00b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56b5fa7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 0x7f56b00b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56b5fa7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56b015ac00 0x7f56b015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56b5fa7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b015b5d0 con 0x7f56b00c6620 2023-10-19T10:36:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56b5fa7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00b45f0 con 0x7f5698005180 2023-10-19T10:36:30.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56b5fa7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00b5f40 con 0x7f56b015ac00 2023-10-19T10:36:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56af7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698005180 0x7f56b00b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56aeffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 0x7f56b00b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56aeffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 0x7f56b00b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46658/0 (socket says 172.21.15.50:46658) 2023-10-19T10:36:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.358+0000 7f56aeffd640 1 -- 172.21.15.50:0/132895096 learned_addr learned my addr 172.21.15.50:0/132895096 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56affff640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56b015ac00 0x7f56b015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:30.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56aeffd640 1 -- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56b015ac00 msgr2=0x7f56b015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:30.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56aeffd640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56b015ac00 0x7f56b015b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56aeffd640 1 -- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698005180 msgr2=0x7f56b00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:30.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56aeffd640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698005180 0x7f56b00b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56aeffd640 1 -- 172.21.15.50:0/132895096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b01f5270 con 0x7f56b00c6620 2023-10-19T10:36:30.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56af7fe640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698005180 0x7f56b00b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f56aeffd640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 0x7f56b00b5a00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56a400f7f0 tx=0x7f56a400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.359+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a40052e0 con 0x7f56b00c6620 2023-10-19T10:36:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.360+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4005440 con 0x7f56b00c6620 2023-10-19T10:36:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.360+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01f5550 con 0x7f56b00c6620 2023-10-19T10:36:30.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.360+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400d950 con 0x7f56b00c6620 2023-10-19T10:36:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.360+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b01f5b10 con 0x7f56b00c6620 2023-10-19T10:36:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.360+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f56a40035f0 con 0x7f56b00c6620 2023-10-19T10:36:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.360+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f56a401d460 con 0x7f56b00c6620 2023-10-19T10:36:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.361+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b0073990 con 0x7f56b00c6620 2023-10-19T10:36:30.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.363+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f56a4003a80 con 0x7f56b00c6620 2023-10-19T10:36:30.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.430+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f56b00cd570 con 0x7f56b00c6620 2023-10-19T10:36:30.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.432+0000 7f5697fff640 1 -- 172.21.15.50:0/132895096 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f56a4003c20 con 0x7f56b00c6620 2023-10-19T10:36:30.435 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2023-10-19T10:36:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.433+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 msgr2=0x7f56b00b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.433+0000 7f56b5fa7640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 0x7f56b00b5a00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56a400f7f0 tx=0x7f56a400fcb0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.433+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 shutdown_connections 2023-10-19T10:36:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.433+0000 7f56b5fa7640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56b015ac00 0x7f56b015b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.433+0000 7f56b5fa7640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56b00c6620 0x7f56b00b5a00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.433+0000 7f56b5fa7640 1 --2- 172.21.15.50:0/132895096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5698005180 0x7f56b00b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.434+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 >> 172.21.15.50:0/132895096 conn(0x7f56b006f3f0 msgr2=0x7f56b015de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.434+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 shutdown_connections 2023-10-19T10:36:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:30.434+0000 7f56b5fa7640 1 -- 172.21.15.50:0/132895096 wait complete. 2023-10-19T10:36:30.446 INFO:tasks.ceph:check_enable_crimson: False 2023-10-19T10:36:30.447 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-19T10:36:30.447 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:30.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-19T10:36:30.479 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:30.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-19T10:36:30.552 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:30.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-19T10:36:30.623 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:36:30.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-19T10:36:30.694 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:30.694 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-19T10:36:30.721 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:30.721 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-19T10:36:30.788 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:30.788 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-19T10:36:30.818 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:30.816+0000 7ff14a232040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-19T10:36:30.845 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:36:30.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-19T10:36:30.853 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:30.850+0000 7f3589a45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-19T10:36:30.872 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:30.870+0000 7ff14a232040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-19T10:36:30.898 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 969faf31-afdd-4ca1-862d-266c0a9d117f 0 2023-10-19T10:36:30.906 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:30.904+0000 7f3589a45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-19T10:36:31.038 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.036+0000 7f2a77a09640 1 Processor -- start 2023-10-19T10:36:31.039 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.036+0000 7f2a77a09640 1 -- start start 2023-10-19T10:36:31.041 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a77a09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.041 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a77a09640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2a70070e50 0x7f2a700cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.041 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a77a09640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a700cb920 con 0x7f2a700cae30 2023-10-19T10:36:31.042 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a77a09640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a700ccd40 con 0x7f2a70070e50 2023-10-19T10:36:31.042 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a77a09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700cd020 con 0x7f2a701593e0 2023-10-19T10:36:31.042 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.043 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42690/0 (socket says 172.21.15.183:42690) 2023-10-19T10:36:31.043 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 -- 172.21.15.183:0/3860904225 learned_addr learned my addr 172.21.15.183:0/3860904225 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.043 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a75fa6640 1 --2- 172.21.15.183:0/3860904225 >> v2:172.21.15.183:3300/0 conn(0x7f2a70070e50 0x7f2a700cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.043 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 -- 172.21.15.183:0/3860904225 >> v2:172.21.15.183:3300/0 conn(0x7f2a70070e50 msgr2=0x7f2a700cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:31.044 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 --2- 172.21.15.183:0/3860904225 >> v2:172.21.15.183:3300/0 conn(0x7f2a70070e50 0x7f2a700cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.044 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 -- 172.21.15.183:0/3860904225 >> v1:172.21.15.50:6789/0 conn(0x7f2a700cae30 legacy=0x7f2a70070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.044 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a757a5640 1 -- 172.21.15.183:0/3860904225 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700cd300 con 0x7f2a701593e0 2023-10-19T10:36:31.044 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.037+0000 7f2a75fa6640 1 --2- 172.21.15.183:0/3860904225 >> v2:172.21.15.183:3300/0 conn(0x7f2a70070e50 0x7f2a700cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:31.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a757a5640 1 --2- 172.21.15.183:0/3860904225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700b1f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009870 tx=0x7f2a6c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4925376f31f253 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a67fff640 1 -- 172.21.15.183:0/3860904225 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c030840 con 0x7f2a701593e0 2023-10-19T10:36:31.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a67fff640 1 -- 172.21.15.183:0/3860904225 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c0309a0 con 0x7f2a701593e0 2023-10-19T10:36:31.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a67fff640 1 -- 172.21.15.183:0/3860904225 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c030d30 con 0x7f2a701593e0 2023-10-19T10:36:31.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a77a09640 1 -- 172.21.15.183:0/3860904225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 msgr2=0x7f2a700b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a77a09640 1 --2- 172.21.15.183:0/3860904225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700b1f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009870 tx=0x7f2a6c02b980 comp rx=0 tx=0).stop 2023-10-19T10:36:31.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.038+0000 7f2a77a09640 1 -- 172.21.15.183:0/3860904225 shutdown_connections 2023-10-19T10:36:31.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.039+0000 7f2a77a09640 1 --2- 172.21.15.183:0/3860904225 >> v2:172.21.15.183:3300/0 conn(0x7f2a70070e50 0x7f2a700cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.039+0000 7f2a77a09640 1 --2- 172.21.15.183:0/3860904225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700b1f60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.039+0000 7f2a77a09640 1 -- 172.21.15.183:0/3860904225 >> 172.21.15.183:0/3860904225 conn(0x7f2a700cbb50 msgr2=0x7f2a700cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.039+0000 7f2a77a09640 1 -- 172.21.15.183:0/3860904225 shutdown_connections 2023-10-19T10:36:31.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.039+0000 7f2a77a09640 1 -- 172.21.15.183:0/3860904225 wait complete. 2023-10-19T10:36:31.049 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 Processor -- start 2023-10-19T10:36:31.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 -- start start 2023-10-19T10:36:31.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a70070e50 0x7f2a70156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 0x7f2a700c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700c2f80 con 0x7f2a70070e50 2023-10-19T10:36:31.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a701ee8b0 con 0x7f2a700cae30 2023-10-19T10:36:31.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a77a09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700c2be0 con 0x7f2a701593e0 2023-10-19T10:36:31.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a74fa4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 0x7f2a700c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a75fa6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.052 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.040+0000 7f2a74fa4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 0x7f2a700c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39650/0 (socket says 172.21.15.183:39650) 2023-10-19T10:36:31.052 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 -- 172.21.15.183:0/1301488490 learned_addr learned my addr 172.21.15.183:0/1301488490 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.052 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a757a5640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a70070e50 0x7f2a70156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.052 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 -- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 msgr2=0x7f2a700c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.052 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.053 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 -- 172.21.15.183:0/1301488490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a70070e50 msgr2=0x7f2a70156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.053 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a70070e50 0x7f2a70156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.053 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 -- 172.21.15.183:0/1301488490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68009a70 con 0x7f2a700cae30 2023-10-19T10:36:31.053 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a74fa4640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 0x7f2a700c9040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000c450 tx=0x7f2a6000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.053 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a600052d0 con 0x7f2a700cae30 2023-10-19T10:36:31.054 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a60005430 con 0x7f2a700cae30 2023-10-19T10:36:31.054 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c0094d0 con 0x7f2a700cae30 2023-10-19T10:36:31.054 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60012870 con 0x7f2a700cae30 2023-10-19T10:36:31.055 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2a60012cd0 con 0x7f2a700cae30 2023-10-19T10:36:31.055 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a700c5850 con 0x7f2a700cae30 2023-10-19T10:36:31.055 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.041+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2a60003930 con 0x7f2a700cae30 2023-10-19T10:36:31.056 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.042+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a700c30c0 con 0x7f2a700cae30 2023-10-19T10:36:31.056 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.044+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2a700c30c0 con 0x7f2a700cae30 2023-10-19T10:36:31.118 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.116+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "969faf31-afdd-4ca1-862d-266c0a9d117f", "id": 0} v 0) v1 -- 0x7f2a70073990 con 0x7f2a700cae30 2023-10-19T10:36:31.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.119+0000 7f2a65ffb640 1 -- 172.21.15.183:0/1301488490 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "969faf31-afdd-4ca1-862d-266c0a9d117f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2a6002a0b0 con 0x7f2a700cae30 2023-10-19T10:36:31.122 INFO:teuthology.orchestra.run.smithi183.stdout:0 2023-10-19T10:36:31.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 msgr2=0x7f2a700c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 0x7f2a700c9040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000c450 tx=0x7f2a6000c910 comp rx=0 tx=0).stop 2023-10-19T10:36:31.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 shutdown_connections 2023-10-19T10:36:31.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2a701593e0 0x7f2a700c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a700cae30 0x7f2a700c9040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 --2- 172.21.15.183:0/1301488490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a70070e50 0x7f2a70156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 >> 172.21.15.183:0/1301488490 conn(0x7f2a700cbb50 msgr2=0x7f2a7006dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 shutdown_connections 2023-10-19T10:36:31.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.120+0000 7f2a77a09640 1 -- 172.21.15.183:0/1301488490 wait complete. 2023-10-19T10:36:31.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 446ff7aa-e634-46bf-aba0-fa0084bc4d57 1 2023-10-19T10:36:31.232 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.230+0000 7f5ae7c77640 1 Processor -- start 2023-10-19T10:36:31.233 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.230+0000 7f5ae7c77640 1 -- start start 2023-10-19T10:36:31.234 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae7c77640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 0x7f5ae00cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.235 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae7c77640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5ae0160cd0 0x7f5ae0160050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.235 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae7c77640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 0x7f5ae015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.235 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae7c77640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae00ae420 con 0x7f5ae0160cd0 2023-10-19T10:36:31.235 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae7c77640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae0160a20 con 0x7f5ae01534f0 2023-10-19T10:36:31.235 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae7c77640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00cb220 con 0x7f5ae0160590 2023-10-19T10:36:31.236 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae5a13640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 0x7f5ae00cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.236 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 0x7f5ae015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.236 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 0x7f5ae015ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42716/0 (socket says 172.21.15.183:42716) 2023-10-19T10:36:31.236 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae5a13640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 0x7f5ae00cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39666/0 (socket says 172.21.15.183:39666) 2023-10-19T10:36:31.236 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 -- 172.21.15.183:0/613884207 learned_addr learned my addr 172.21.15.183:0/613884207 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.237 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae5212640 1 --2- 172.21.15.183:0/613884207 >> v2:172.21.15.50:3300/0 conn(0x7f5ae0160cd0 0x7f5ae0160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.237 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 -- 172.21.15.183:0/613884207 >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 msgr2=0x7f5ae00cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:31.237 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 --2- 172.21.15.183:0/613884207 >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 0x7f5ae00cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.237 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 -- 172.21.15.183:0/613884207 >> v2:172.21.15.50:3300/0 conn(0x7f5ae0160cd0 msgr2=0x7f5ae0160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:31.237 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 --2- 172.21.15.183:0/613884207 >> v2:172.21.15.50:3300/0 conn(0x7f5ae0160cd0 0x7f5ae0160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.237 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae6214640 1 -- 172.21.15.183:0/613884207 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae006d0a0 con 0x7f5ae0160590 2023-10-19T10:36:31.238 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.231+0000 7f5ae5a13640 1 --2- 172.21.15.183:0/613884207 >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 0x7f5ae00cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:31.238 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.232+0000 7f5ae6214640 1 --2- 172.21.15.183:0/613884207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 0x7f5ae015ca60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009e50 tx=0x7f5ad802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55fd67af877b7ca server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.238 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.232+0000 7f5ae4a11640 1 -- 172.21.15.183:0/613884207 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad802ca70 con 0x7f5ae0160590 2023-10-19T10:36:31.238 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.232+0000 7f5ae4a11640 1 -- 172.21.15.183:0/613884207 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad802cbd0 con 0x7f5ae0160590 2023-10-19T10:36:31.238 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.232+0000 7f5ae4a11640 1 -- 172.21.15.183:0/613884207 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80343f0 con 0x7f5ae0160590 2023-10-19T10:36:31.241 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 -- 172.21.15.183:0/613884207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 msgr2=0x7f5ae015ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.241 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/613884207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 0x7f5ae015ca60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009e50 tx=0x7f5ad802bed0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.242 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 -- 172.21.15.183:0/613884207 shutdown_connections 2023-10-19T10:36:31.242 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/613884207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160590 0x7f5ae015ca60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.242 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/613884207 >> v2:172.21.15.50:3300/0 conn(0x7f5ae0160cd0 0x7f5ae0160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.242 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/613884207 >> v2:172.21.15.183:3300/0 conn(0x7f5ae01534f0 0x7f5ae00cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.242 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 -- 172.21.15.183:0/613884207 >> 172.21.15.183:0/613884207 conn(0x7f5ae00cb390 msgr2=0x7f5ae00cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.243 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 -- 172.21.15.183:0/613884207 shutdown_connections 2023-10-19T10:36:31.243 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.233+0000 7f5ae7c77640 1 -- 172.21.15.183:0/613884207 wait complete. 2023-10-19T10:36:31.243 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.234+0000 7f5ae7c77640 1 Processor -- start 2023-10-19T10:36:31.243 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.234+0000 7f5ae7c77640 1 -- start start 2023-10-19T10:36:31.243 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae7c77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae01534f0 0x7f5ae0140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.244 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae7c77640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 0x7f5ae014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.244 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae7c77640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 0x7f5ae0150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.244 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae7c77640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00b1f70 con 0x7f5ae01534f0 2023-10-19T10:36:31.244 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae7c77640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0140890 con 0x7f5ae0160590 2023-10-19T10:36:31.244 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae7c77640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01f6220 con 0x7f5ae0160cd0 2023-10-19T10:36:31.245 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 0x7f5ae0150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.245 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae5212640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 0x7f5ae014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.245 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae5212640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 0x7f5ae014fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39676/0 (socket says 172.21.15.183:39676) 2023-10-19T10:36:31.245 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 0x7f5ae0150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42720/0 (socket says 172.21.15.183:42720) 2023-10-19T10:36:31.245 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae5212640 1 -- 172.21.15.183:0/3766388913 learned_addr learned my addr 172.21.15.183:0/3766388913 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.245 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae5a13640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae01534f0 0x7f5ae0140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.246 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 -- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 msgr2=0x7f5ae014fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.246 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 0x7f5ae014fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.246 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 -- 172.21.15.183:0/3766388913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae01534f0 msgr2=0x7f5ae0140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.246 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae01534f0 0x7f5ae0140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.246 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 -- 172.21.15.183:0/3766388913 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc009400 con 0x7f5ae0160cd0 2023-10-19T10:36:31.247 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae5212640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 0x7f5ae014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:31.247 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.235+0000 7f5ae6214640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 0x7f5ae0150330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00cb360 tx=0x7f5ad8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.247 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8009510 con 0x7f5ae0160cd0 2023-10-19T10:36:31.247 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad803e450 con 0x7f5ae0160cd0 2023-10-19T10:36:31.247 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad8009b30 con 0x7f5ae0160cd0 2023-10-19T10:36:31.248 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad803e820 con 0x7f5ae0160cd0 2023-10-19T10:36:31.248 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae00b1b90 con 0x7f5ae0160cd0 2023-10-19T10:36:31.248 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5ad803ec80 con 0x7f5ae0160cd0 2023-10-19T10:36:31.248 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f5ad8048da0 con 0x7f5ae0160cd0 2023-10-19T10:36:31.248 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.236+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae00b20b0 con 0x7f5ae0160cd0 2023-10-19T10:36:31.248 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.239+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5ae00b20b0 con 0x7f5ae0160cd0 2023-10-19T10:36:31.312 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.310+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "446ff7aa-e634-46bf-aba0-fa0084bc4d57", "id": 1} v 0) v1 -- 0x7f5ae0073990 con 0x7f5ae0160cd0 2023-10-19T10:36:31.315 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.313+0000 7f5ad67fc640 1 -- 172.21.15.183:0/3766388913 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "446ff7aa-e634-46bf-aba0-fa0084bc4d57", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5ad8043020 con 0x7f5ae0160cd0 2023-10-19T10:36:31.315 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-10-19T10:36:31.316 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 msgr2=0x7f5ae0150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.316 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 0x7f5ae0150330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00cb360 tx=0x7f5ad8004a40 comp rx=0 tx=0).stop 2023-10-19T10:36:31.317 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 shutdown_connections 2023-10-19T10:36:31.317 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ae0160cd0 0x7f5ae0150330 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.317 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae0160590 0x7f5ae014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.317 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 --2- 172.21.15.183:0/3766388913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae01534f0 0x7f5ae0140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.317 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 >> 172.21.15.183:0/3766388913 conn(0x7f5ae00cb390 msgr2=0x7f5ae00cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.318 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 shutdown_connections 2023-10-19T10:36:31.318 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.314+0000 7f5ae7c77640 1 -- 172.21.15.183:0/3766388913 wait complete. 2023-10-19T10:36:31.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 6852e173-ea11-407a-bc20-c9f6669740c3 2 2023-10-19T10:36:31.426 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.424+0000 7fa428d02640 1 Processor -- start 2023-10-19T10:36:31.427 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.424+0000 7fa428d02640 1 -- start start 2023-10-19T10:36:31.429 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.424+0000 7fa428d02640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa42406b9a0 0x7fa4240ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.429 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.424+0000 7fa428d02640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4240af5f0 con 0x7fa4240d0d20 2023-10-19T10:36:31.429 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.424+0000 7fa428d02640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4240ca8e0 con 0x7fa42415c6d0 2023-10-19T10:36:31.429 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.424+0000 7fa428d02640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4240af310 con 0x7fa42406b9a0 2023-10-19T10:36:31.430 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa42259c640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fa42415c6d0 0x7fa42406b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:52382/0 (socket says 172.21.15.183:52382) 2023-10-19T10:36:31.430 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa42259c640 1 -- 172.21.15.183:0/3406215443 learned_addr learned my addr 172.21.15.183:0/3406215443 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-10-19T10:36:31.430 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa421d9b640 1 --2- 172.21.15.183:0/3406215443 >> v2:172.21.15.50:3300/0 conn(0x7fa42406b9a0 0x7fa4240ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.430 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa42159a640 1 -- 172.21.15.183:0/3406215443 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252144148 0 0) 0x7fa4240ca8e0 con 0x7fa42415c6d0 2023-10-19T10:36:31.431 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa42159a640 1 -- 172.21.15.183:0/3406215443 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003520 con 0x7fa42415c6d0 2023-10-19T10:36:31.431 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa421d9b640 1 -- 172.21.15.183:0/3406215443 >> v1:172.21.15.183:6789/0 conn(0x7fa42415c6d0 legacy=0x7fa42406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.431 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa42159a640 1 -- 172.21.15.183:0/3406215443 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3989643409 0 0) 0x7fa40c003520 con 0x7fa42415c6d0 2023-10-19T10:36:31.431 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa421d9b640 1 -- 172.21.15.183:0/3406215443 >> v1:172.21.15.50:6789/0 conn(0x7fa4240d0d20 legacy=0x7fa4240cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.432 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.425+0000 7fa421d9b640 1 -- 172.21.15.183:0/3406215443 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42406ac50 con 0x7fa42406b9a0 2023-10-19T10:36:31.432 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.426+0000 7fa421d9b640 1 --2- 172.21.15.183:0/3406215443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42406b9a0 0x7fa4240ce530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa4180097a0 tx=0x7fa41802b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe725f781415141 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.432 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.426+0000 7fa42159a640 1 -- 172.21.15.183:0/3406215443 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414001670 con 0x7fa42406b9a0 2023-10-19T10:36:31.432 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.426+0000 7fa42159a640 1 -- 172.21.15.183:0/3406215443 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41802e050 con 0x7fa42406b9a0 2023-10-19T10:36:31.432 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.426+0000 7fa42159a640 1 -- 172.21.15.183:0/3406215443 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418030730 con 0x7fa42406b9a0 2023-10-19T10:36:31.435 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 -- 172.21.15.183:0/3406215443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42406b9a0 msgr2=0x7fa4240ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.436 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 --2- 172.21.15.183:0/3406215443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42406b9a0 0x7fa4240ce530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa4180097a0 tx=0x7fa41802b680 comp rx=0 tx=0).stop 2023-10-19T10:36:31.436 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 -- 172.21.15.183:0/3406215443 shutdown_connections 2023-10-19T10:36:31.436 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 --2- 172.21.15.183:0/3406215443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42406b9a0 0x7fa4240ce530 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.436 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 -- 172.21.15.183:0/3406215443 >> 172.21.15.183:0/3406215443 conn(0x7fa4240d0200 msgr2=0x7fa4240d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.436 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 -- 172.21.15.183:0/3406215443 shutdown_connections 2023-10-19T10:36:31.437 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.427+0000 7fa428d02640 1 -- 172.21.15.183:0/3406215443 wait complete. 2023-10-19T10:36:31.437 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.428+0000 7fa428d02640 1 Processor -- start 2023-10-19T10:36:31.437 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.428+0000 7fa428d02640 1 -- start start 2023-10-19T10:36:31.437 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa428d02640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 0x7fa4241580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.438 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa428d02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 0x7fa4240b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.438 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa428d02640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42415c6d0 0x7fa4240b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.438 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa428d02640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4240c8a90 con 0x7fa42415c6d0 2023-10-19T10:36:31.438 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa428d02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4241f1d60 con 0x7fa4240d0d20 2023-10-19T10:36:31.438 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa428d02640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4240c7e80 con 0x7fa42406b9a0 2023-10-19T10:36:31.439 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa421d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 0x7fa4240b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.439 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 0x7fa4241580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.439 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa421d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 0x7fa4240b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39686/0 (socket says 172.21.15.183:39686) 2023-10-19T10:36:31.439 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 0x7fa4241580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42732/0 (socket says 172.21.15.183:42732) 2023-10-19T10:36:31.439 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa421d9b640 1 -- 172.21.15.183:0/2727531401 learned_addr learned my addr 172.21.15.183:0/2727531401 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.440 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa422d9d640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42415c6d0 0x7fa4240b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.440 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 -- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 msgr2=0x7fa4240b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.440 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 0x7fa4240b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.440 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 -- 172.21.15.183:0/2727531401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42415c6d0 msgr2=0x7fa4240b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.440 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42415c6d0 0x7fa4240b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.440 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa42259c640 1 -- 172.21.15.183:0/2727531401 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009400 con 0x7fa42406b9a0 2023-10-19T10:36:31.441 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.429+0000 7fa421d9b640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 0x7fa4240b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:31.441 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa42259c640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 0x7fa4241580a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa414005880 tx=0x7fa41400c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.441 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414005ce0 con 0x7fa42406b9a0 2023-10-19T10:36:31.441 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa414005e60 con 0x7fa42406b9a0 2023-10-19T10:36:31.442 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140072b0 con 0x7fa42406b9a0 2023-10-19T10:36:31.442 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4240c8160 con 0x7fa42406b9a0 2023-10-19T10:36:31.442 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4240c8720 con 0x7fa42406b9a0 2023-10-19T10:36:31.442 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa41401c440 con 0x7fa42406b9a0 2023-10-19T10:36:31.443 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.430+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa41401cb10 con 0x7fa42406b9a0 2023-10-19T10:36:31.443 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.431+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa424073990 con 0x7fa42406b9a0 2023-10-19T10:36:31.443 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.433+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa414021060 con 0x7fa42406b9a0 2023-10-19T10:36:31.444 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.434+0000 7ff14a232040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-19T10:36:31.489 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.487+0000 7ff14a232040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-19T10:36:31.506 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.503+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6852e173-ea11-407a-bc20-c9f6669740c3", "id": 2} v 0) v1 -- 0x7fa42406a910 con 0x7fa42406b9a0 2023-10-19T10:36:31.509 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa412ffd640 1 -- 172.21.15.183:0/2727531401 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6852e173-ea11-407a-bc20-c9f6669740c3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa414017070 con 0x7fa42406b9a0 2023-10-19T10:36:31.509 INFO:teuthology.orchestra.run.smithi183.stdout:2 2023-10-19T10:36:31.510 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 msgr2=0x7fa4241580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.511 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 0x7fa4241580a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa414005880 tx=0x7fa41400c9b0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.511 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 shutdown_connections 2023-10-19T10:36:31.511 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42415c6d0 0x7fa4240b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.511 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4240d0d20 0x7fa4240b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.512 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 --2- 172.21.15.183:0/2727531401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42406b9a0 0x7fa4241580a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.512 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 >> 172.21.15.183:0/2727531401 conn(0x7fa4240d0200 msgr2=0x7fa4240b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.512 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 shutdown_connections 2023-10-19T10:36:31.512 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.507+0000 7fa428d02640 1 -- 172.21.15.183:0/2727531401 wait complete. 2023-10-19T10:36:31.520 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new b4f1c130-4798-40fa-a122-8b38a2ea0d8b 3 2023-10-19T10:36:31.536 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.534+0000 7ff14a232040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-19T10:36:31.577 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.575+0000 7f3589a45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-19T10:36:31.592 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.590+0000 7ff14a232040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-19T10:36:31.620 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.617+0000 7fbbe8143640 1 Processor -- start 2023-10-19T10:36:31.620 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.617+0000 7fbbe8143640 1 -- start start 2023-10-19T10:36:31.621 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe8143640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 0x7fbbe00d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.622 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe8143640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe015de50 con 0x7fbbe015da70 2023-10-19T10:36:31.622 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe8143640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00cec30 con 0x7fbbe00c45f0 2023-10-19T10:36:31.622 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe8143640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00c41d0 con 0x7fbbe00cd300 2023-10-19T10:36:31.622 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe56de640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 0x7fbbe00d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.622 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe56de640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 0x7fbbe00d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42748/0 (socket says 172.21.15.183:42748) 2023-10-19T10:36:31.623 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe56de640 1 -- 172.21.15.183:0/1914660760 learned_addr learned my addr 172.21.15.183:0/1914660760 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.623 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.618+0000 7fbbe66e0640 1 --1- 172.21.15.183:0/1914660760 >> v1:172.21.15.183:6789/0 conn(0x7fbbe00c45f0 0x7fbbe00c4a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:52386/0 (socket says 172.21.15.183:52386) 2023-10-19T10:36:31.623 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.619+0000 7fbbe56de640 1 -- 172.21.15.183:0/1914660760 >> v1:172.21.15.183:6789/0 conn(0x7fbbe00c45f0 legacy=0x7fbbe00c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.623 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.619+0000 7fbbe56de640 1 -- 172.21.15.183:0/1914660760 >> v1:172.21.15.50:6789/0 conn(0x7fbbe015da70 legacy=0x7fbbe00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.623 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.619+0000 7fbbe56de640 1 -- 172.21.15.183:0/1914660760 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00cef10 con 0x7fbbe00cd300 2023-10-19T10:36:31.623 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.619+0000 7fbbe4edd640 1 -- 172.21.15.183:0/1914660760 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431512258 0 0) 0x7fbbe00cec30 con 0x7fbbe00c45f0 2023-10-19T10:36:31.627 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.619+0000 7fbbe56de640 1 --2- 172.21.15.183:0/1914660760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 0x7fbbe00d0d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0098a0 tx=0x7fbbdc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d708124d49d4e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.627 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe4edd640 1 -- 172.21.15.183:0/1914660760 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc030840 con 0x7fbbe00cd300 2023-10-19T10:36:31.627 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe4edd640 1 -- 172.21.15.183:0/1914660760 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc0309a0 con 0x7fbbe00cd300 2023-10-19T10:36:31.627 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe4edd640 1 -- 172.21.15.183:0/1914660760 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc030d30 con 0x7fbbe00cd300 2023-10-19T10:36:31.628 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 -- 172.21.15.183:0/1914660760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 msgr2=0x7fbbe00d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.628 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 --2- 172.21.15.183:0/1914660760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 0x7fbbe00d0d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0098a0 tx=0x7fbbdc02b980 comp rx=0 tx=0).stop 2023-10-19T10:36:31.628 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 -- 172.21.15.183:0/1914660760 shutdown_connections 2023-10-19T10:36:31.628 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 --2- 172.21.15.183:0/1914660760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00cd300 0x7fbbe00d0d20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.628 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 -- 172.21.15.183:0/1914660760 >> 172.21.15.183:0/1914660760 conn(0x7fbbe00d0200 msgr2=0x7fbbe00d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.629 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 -- 172.21.15.183:0/1914660760 shutdown_connections 2023-10-19T10:36:31.629 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.620+0000 7fbbe8143640 1 -- 172.21.15.183:0/1914660760 wait complete. 2023-10-19T10:36:31.629 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.621+0000 7fbbe8143640 1 Processor -- start 2023-10-19T10:36:31.629 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.621+0000 7fbbe8143640 1 -- start start 2023-10-19T10:36:31.629 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe8143640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 0x7fbbe00c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.629 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe8143640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbe00cd300 0x7fbbe01587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe8143640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 0x7fbbe006eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe8143640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe006c980 con 0x7fbbe00cd300 2023-10-19T10:36:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe8143640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0158cf0 con 0x7fbbe015da70 2023-10-19T10:36:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe8143640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0159bd0 con 0x7fbbe00c45f0 2023-10-19T10:36:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe5edf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 0x7fbbe00c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe66e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 0x7fbbe006eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe5edf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 0x7fbbe00c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42750/0 (socket says 172.21.15.183:42750) 2023-10-19T10:36:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe5edf640 1 -- 172.21.15.183:0/1355315596 learned_addr learned my addr 172.21.15.183:0/1355315596 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe66e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 0x7fbbe006eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39694/0 (socket says 172.21.15.183:39694) 2023-10-19T10:36:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.622+0000 7fbbe56de640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbe00cd300 0x7fbbe01587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe5edf640 1 -- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 msgr2=0x7fbbe006eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe5edf640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 0x7fbbe006eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe5edf640 1 -- 172.21.15.183:0/1355315596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbe00cd300 msgr2=0x7fbbe01587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe5edf640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbe00cd300 0x7fbbe01587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe5edf640 1 -- 172.21.15.183:0/1355315596 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0094d0 con 0x7fbbe00c45f0 2023-10-19T10:36:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe66e0640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 0x7fbbe006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:31.633 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe5edf640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 0x7fbbe00c2e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c400 tx=0x7fbbd000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.633 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00052d0 con 0x7fbbe00c45f0 2023-10-19T10:36:31.633 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0005430 con 0x7fbbe00c45f0 2023-10-19T10:36:31.633 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe0159630 con 0x7fbbe00c45f0 2023-10-19T10:36:31.633 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0003830 con 0x7fbbe00c45f0 2023-10-19T10:36:31.634 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01598f0 con 0x7fbbe00c45f0 2023-10-19T10:36:31.634 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbbd0003c90 con 0x7fbbe00c45f0 2023-10-19T10:36:31.634 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.623+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fbbd00128b0 con 0x7fbbe00c45f0 2023-10-19T10:36:31.634 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.624+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe00cf1a0 con 0x7fbbe00c45f0 2023-10-19T10:36:31.635 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.629+0000 7f3589a45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-19T10:36:31.635 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.626+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbbd0017070 con 0x7fbbe00c45f0 2023-10-19T10:36:31.637 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.635+0000 7ff14a232040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-19T10:36:31.679 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.676+0000 7f3589a45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-19T10:36:31.700 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.698+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b4f1c130-4798-40fa-a122-8b38a2ea0d8b", "id": 3} v 0) v1 -- 0x7fbbe0073990 con 0x7fbbe00c45f0 2023-10-19T10:36:31.703 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.701+0000 7fbbd67fc640 1 -- 172.21.15.183:0/1355315596 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4f1c130-4798-40fa-a122-8b38a2ea0d8b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbbd0012420 con 0x7fbbe00c45f0 2023-10-19T10:36:31.703 INFO:teuthology.orchestra.run.smithi183.stdout:3 2023-10-19T10:36:31.704 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 msgr2=0x7fbbe00c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.705 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 0x7fbbe00c2e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c400 tx=0x7fbbd000c8c0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.705 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 shutdown_connections 2023-10-19T10:36:31.705 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe015da70 0x7fbbe006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.705 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbe00cd300 0x7fbbe01587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.705 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 --2- 172.21.15.183:0/1355315596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbe00c45f0 0x7fbbe00c2e20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.706 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 >> 172.21.15.183:0/1355315596 conn(0x7fbbe00d0200 msgr2=0x7fbbe00cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.706 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 shutdown_connections 2023-10-19T10:36:31.706 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.702+0000 7fbbe8143640 1 -- 172.21.15.183:0/1355315596 wait complete. 2023-10-19T10:36:31.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new dbc106ec-a15b-4b9c-82f1-b6e1cefceef0 4 2023-10-19T10:36:31.737 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.732+0000 7f3589a45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-19T10:36:31.742 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.740+0000 7ff14a232040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-19T10:36:31.779 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.776+0000 7f3589a45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-19T10:36:31.816 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.813+0000 7ff14a232040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-19T10:36:31.841 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.839+0000 7fa29309d640 1 Processor -- start 2023-10-19T10:36:31.842 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.839+0000 7fa29309d640 1 -- start start 2023-10-19T10:36:31.844 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29309d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa28c15a4f0 0x7fa28c069800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.844 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29309d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c069d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.844 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29309d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c0cb990 con 0x7fa28c0cb4b0 2023-10-19T10:36:31.844 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29309d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c070870 con 0x7fa28c15a4f0 2023-10-19T10:36:31.844 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29309d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c070b50 con 0x7fa28c06ee50 2023-10-19T10:36:31.845 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.845 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c069d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42766/0 (socket says 172.21.15.183:42766) 2023-10-19T10:36:31.845 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 -- 172.21.15.183:0/202931294 learned_addr learned my addr 172.21.15.183:0/202931294 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.845 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa290e39640 1 --2- 172.21.15.183:0/202931294 >> v2:172.21.15.50:3300/0 conn(0x7fa28c15a4f0 0x7fa28c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.845 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 -- 172.21.15.183:0/202931294 >> v2:172.21.15.50:3300/0 conn(0x7fa28c15a4f0 msgr2=0x7fa28c069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:31.846 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 --2- 172.21.15.183:0/202931294 >> v2:172.21.15.50:3300/0 conn(0x7fa28c15a4f0 0x7fa28c069800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.846 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 -- 172.21.15.183:0/202931294 >> v1:172.21.15.50:6789/0 conn(0x7fa28c0cb4b0 legacy=0x7fa28c06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.846 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.840+0000 7fa29163a640 1 -- 172.21.15.183:0/202931294 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c070e30 con 0x7fa28c06ee50 2023-10-19T10:36:31.846 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.841+0000 7fa29163a640 1 --2- 172.21.15.183:0/202931294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c069d40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa284009e50 tx=0x7fa28402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e524df8c1b310dc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.846 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.841+0000 7fa28b7fe640 1 -- 172.21.15.183:0/202931294 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28402ca70 con 0x7fa28c06ee50 2023-10-19T10:36:31.847 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.841+0000 7fa28b7fe640 1 -- 172.21.15.183:0/202931294 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28402cbd0 con 0x7fa28c06ee50 2023-10-19T10:36:31.847 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.841+0000 7fa28b7fe640 1 -- 172.21.15.183:0/202931294 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840343f0 con 0x7fa28c06ee50 2023-10-19T10:36:31.850 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 -- 172.21.15.183:0/202931294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 msgr2=0x7fa28c069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.850 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 --2- 172.21.15.183:0/202931294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c069d40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa284009e50 tx=0x7fa28402bed0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.850 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 -- 172.21.15.183:0/202931294 shutdown_connections 2023-10-19T10:36:31.850 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 --2- 172.21.15.183:0/202931294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c069d40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.851 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 --2- 172.21.15.183:0/202931294 >> v2:172.21.15.50:3300/0 conn(0x7fa28c15a4f0 0x7fa28c069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.851 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 -- 172.21.15.183:0/202931294 >> 172.21.15.183:0/202931294 conn(0x7fa28c0cbb50 msgr2=0x7fa28c0cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.851 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 -- 172.21.15.183:0/202931294 shutdown_connections 2023-10-19T10:36:31.851 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.842+0000 7fa29309d640 1 -- 172.21.15.183:0/202931294 wait complete. 2023-10-19T10:36:31.851 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.843+0000 7fa29309d640 1 Processor -- start 2023-10-19T10:36:31.852 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.843+0000 7fa29309d640 1 -- start start 2023-10-19T10:36:31.852 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.852 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa28c0cb4b0 0x7fa28c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.852 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c15a4f0 0x7fa28c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:31.852 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0ac620 con 0x7fa28c15a4f0 2023-10-19T10:36:31.853 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0aba10 con 0x7fa28c0cb4b0 2023-10-19T10:36:31.853 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0abcf0 con 0x7fa28c06ee50 2023-10-19T10:36:31.853 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.853 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c15d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42776/0 (socket says 172.21.15.183:42776) 2023-10-19T10:36:31.853 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 -- 172.21.15.183:0/201759475 learned_addr learned my addr 172.21.15.183:0/201759475 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:31.854 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa28bfff640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa28c0cb4b0 0x7fa28c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.854 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29163a640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c15a4f0 0x7fa28c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:31.854 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 -- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa28c0cb4b0 msgr2=0x7fa28c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.854 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa28c0cb4b0 0x7fa28c0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.854 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 -- 172.21.15.183:0/201759475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c15a4f0 msgr2=0x7fa28c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.855 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c15a4f0 0x7fa28c0cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.855 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 -- 172.21.15.183:0/201759475 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284009b30 con 0x7fa28c06ee50 2023-10-19T10:36:31.855 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa290e39640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c15d3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa28000e870 tx=0x7fa28000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:31.855 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2800094f0 con 0x7fa28c06ee50 2023-10-19T10:36:31.856 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28000c940 con 0x7fa28c06ee50 2023-10-19T10:36:31.856 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28000ccf0 con 0x7fa28c06ee50 2023-10-19T10:36:31.856 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.844+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c15aa50 con 0x7fa28c06ee50 2023-10-19T10:36:31.856 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.845+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c0ac2b0 con 0x7fa28c06ee50 2023-10-19T10:36:31.856 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.845+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa280005c90 con 0x7fa28c06ee50 2023-10-19T10:36:31.857 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.845+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa2800067f0 con 0x7fa28c06ee50 2023-10-19T10:36:31.857 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.846+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c06fff0 con 0x7fa28c06ee50 2023-10-19T10:36:31.857 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.848+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa280021060 con 0x7fa28c06ee50 2023-10-19T10:36:31.864 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.862+0000 7ff14a232040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-19T10:36:31.884 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.881+0000 7f3589a45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-19T10:36:31.918 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.915+0000 7ff14a232040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-19T10:36:31.922 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.919+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dbc106ec-a15b-4b9c-82f1-b6e1cefceef0", "id": 4} v 0) v1 -- 0x7fa28c073990 con 0x7fa28c06ee50 2023-10-19T10:36:31.925 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.922+0000 7fa2897fa640 1 -- 172.21.15.183:0/201759475 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dbc106ec-a15b-4b9c-82f1-b6e1cefceef0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa280017070 con 0x7fa28c06ee50 2023-10-19T10:36:31.925 INFO:teuthology.orchestra.run.smithi183.stdout:4 2023-10-19T10:36:31.926 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 msgr2=0x7fa28c15d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:31.926 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c15d3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa28000e870 tx=0x7fa28000ed30 comp rx=0 tx=0).stop 2023-10-19T10:36:31.927 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 shutdown_connections 2023-10-19T10:36:31.927 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c15a4f0 0x7fa28c0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.927 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa28c0cb4b0 0x7fa28c0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.927 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 --2- 172.21.15.183:0/201759475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa28c06ee50 0x7fa28c15d3f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:31.927 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 >> 172.21.15.183:0/201759475 conn(0x7fa28c0cbb50 msgr2=0x7fa28c0afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:31.928 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 shutdown_connections 2023-10-19T10:36:31.928 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:31.923+0000 7fa29309d640 1 -- 172.21.15.183:0/201759475 wait complete. 2023-10-19T10:36:31.936 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new de88c853-3324-48a2-a105-bf15a3dea21c 5 2023-10-19T10:36:31.957 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:31.955+0000 7f3589a45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-19T10:36:31.968 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:31.965+0000 7ff14a232040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-19T10:36:32.005 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.002+0000 7f3589a45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-19T10:36:32.035 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.033+0000 7fcf7b5eb640 1 Processor -- start 2023-10-19T10:36:32.036 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.033+0000 7fcf7b5eb640 1 -- start start 2023-10-19T10:36:32.037 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf7b5eb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf740cd3b0 0x7fcf7415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.037 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf7b5eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 0x7fcf740698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.038 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf7b5eb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf74160e40 con 0x7fcf7415a0b0 2023-10-19T10:36:32.038 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf7b5eb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740cee50 con 0x7fcf740cd3b0 2023-10-19T10:36:32.038 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf7b5eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7415a490 con 0x7fcf74070740 2023-10-19T10:36:32.038 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf79b88640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 0x7fcf740698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.039 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf78b86640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf740cd3b0 0x7fcf7415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.039 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf79b88640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 0x7fcf740698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42780/0 (socket says 172.21.15.183:42780) 2023-10-19T10:36:32.039 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf78b86640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf740cd3b0 0x7fcf7415cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39706/0 (socket says 172.21.15.183:39706) 2023-10-19T10:36:32.039 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf79b88640 1 -- 172.21.15.183:0/1311901166 learned_addr learned my addr 172.21.15.183:0/1311901166 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:32.039 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf78b86640 1 -- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 msgr2=0x7fcf740698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:32.040 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf78b86640 1 --2- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 0x7fcf740698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.040 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf78b86640 1 -- 172.21.15.183:0/1311901166 >> v1:172.21.15.183:6789/0 conn(0x7fcf7415a0b0 legacy=0x7fcf740ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.040 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf78b86640 1 -- 172.21.15.183:0/1311901166 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74160230 con 0x7fcf740cd3b0 2023-10-19T10:36:32.040 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.034+0000 7fcf79b88640 1 --2- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 0x7fcf740698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:32.040 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.035+0000 7fcf78b86640 1 --2- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf740cd3b0 0x7fcf7415cae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf640097a0 tx=0x7fcf6402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5255539d769c5b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.040 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.035+0000 7fcf6bfff640 1 -- 172.21.15.183:0/1311901166 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64031780 con 0x7fcf740cd3b0 2023-10-19T10:36:32.041 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.035+0000 7fcf6bfff640 1 -- 172.21.15.183:0/1311901166 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf640318e0 con 0x7fcf740cd3b0 2023-10-19T10:36:32.041 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.035+0000 7fcf6bfff640 1 -- 172.21.15.183:0/1311901166 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64031c70 con 0x7fcf740cd3b0 2023-10-19T10:36:32.044 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf740cd3b0 msgr2=0x7fcf7415cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.044 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf740cd3b0 0x7fcf7415cae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf640097a0 tx=0x7fcf6402b820 comp rx=0 tx=0).stop 2023-10-19T10:36:32.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/1311901166 shutdown_connections 2023-10-19T10:36:32.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf74070740 0x7fcf740698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/1311901166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf740cd3b0 0x7fcf7415cae0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.045 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/1311901166 >> 172.21.15.183:0/1311901166 conn(0x7fcf740cbb50 msgr2=0x7fcf740cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/1311901166 shutdown_connections 2023-10-19T10:36:32.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.036+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/1311901166 wait complete. 2023-10-19T10:36:32.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.037+0000 7fcf7b5eb640 1 Processor -- start 2023-10-19T10:36:32.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.037+0000 7fcf7b5eb640 1 -- start start 2023-10-19T10:36:32.046 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf74070740 0x7fcf740c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf740cd3b0 0x7fcf74151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7415a0b0 0x7fcf7415f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7406f110 con 0x7fcf740cd3b0 2023-10-19T10:36:32.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c4ba0 con 0x7fcf74070740 2023-10-19T10:36:32.047 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740b1c90 con 0x7fcf7415a0b0 2023-10-19T10:36:32.048 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf74070740 0x7fcf740c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.048 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79b88640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7415a0b0 0x7fcf7415f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.048 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79b88640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7415a0b0 0x7fcf7415f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:42782/0 (socket says 172.21.15.183:42782) 2023-10-19T10:36:32.048 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79b88640 1 -- 172.21.15.183:0/3532600434 learned_addr learned my addr 172.21.15.183:0/3532600434 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:32.048 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf78b86640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf740cd3b0 0x7fcf74151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.048 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 -- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7415a0b0 msgr2=0x7fcf7415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.049 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7415a0b0 0x7fcf7415f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.049 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 -- 172.21.15.183:0/3532600434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf740cd3b0 msgr2=0x7fcf74151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.049 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf740cd3b0 0x7fcf74151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.049 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 -- 172.21.15.183:0/3532600434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009b30 con 0x7fcf74070740 2023-10-19T10:36:32.049 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf78b86640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf740cd3b0 0x7fcf74151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:32.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf79387640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf74070740 0x7fcf740c4660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000c3c0 tx=0x7fcf7000c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700052d0 con 0x7fcf74070740 2023-10-19T10:36:32.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf70005430 con 0x7fcf74070740 2023-10-19T10:36:32.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70003830 con 0x7fcf74070740 2023-10-19T10:36:32.050 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.038+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf64009400 con 0x7fcf74070740 2023-10-19T10:36:32.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.039+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcf70003c90 con 0x7fcf74070740 2023-10-19T10:36:32.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.039+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740b21c0 con 0x7fcf74070740 2023-10-19T10:36:32.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.039+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fcf70012420 con 0x7fcf74070740 2023-10-19T10:36:32.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.040+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf74073990 con 0x7fcf74070740 2023-10-19T10:36:32.051 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.042+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcf70017070 con 0x7fcf74070740 2023-10-19T10:36:32.059 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.057+0000 7f3589a45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-19T10:36:32.108 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.105+0000 7f3589a45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-19T10:36:32.115 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.113+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "de88c853-3324-48a2-a105-bf15a3dea21c", "id": 5} v 0) v1 -- 0x7fcf740cec10 con 0x7fcf74070740 2023-10-19T10:36:32.118 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.116+0000 7fcf69ffb640 1 -- 172.21.15.183:0/3532600434 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de88c853-3324-48a2-a105-bf15a3dea21c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf70011e50 con 0x7fcf74070740 2023-10-19T10:36:32.119 INFO:teuthology.orchestra.run.smithi183.stdout:5 2023-10-19T10:36:32.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf74070740 msgr2=0x7fcf740c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf74070740 0x7fcf740c4660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000c3c0 tx=0x7fcf7000c7c0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 shutdown_connections 2023-10-19T10:36:32.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7415a0b0 0x7fcf7415f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf740cd3b0 0x7fcf74151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 --2- 172.21.15.183:0/3532600434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf74070740 0x7fcf740c4660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 >> 172.21.15.183:0/3532600434 conn(0x7fcf740cbb50 msgr2=0x7fcf74154a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 shutdown_connections 2023-10-19T10:36:32.122 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.117+0000 7fcf7b5eb640 1 -- 172.21.15.183:0/3532600434 wait complete. 2023-10-19T10:36:32.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 2acb3a97-1331-427e-a927-b37554c81873 6 2023-10-19T10:36:32.270 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.268+0000 7f6bd5612640 1 Processor -- start 2023-10-19T10:36:32.271 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.268+0000 7f6bd5612640 1 -- start start 2023-10-19T10:36:32.274 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.268+0000 7f6bd5612640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6bd0160220 0x7f6bd00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.274 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.268+0000 7f6bd5612640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd00d06d0 con 0x7f6bd015c690 2023-10-19T10:36:32.274 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.268+0000 7f6bd5612640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd00b7ff0 con 0x7f6bd00cfa60 2023-10-19T10:36:32.275 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.268+0000 7f6bd5612640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd0160700 con 0x7f6bd0160220 2023-10-19T10:36:32.275 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f6bd00cfa60 0x7f6bd00d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:52402/0 (socket says 172.21.15.183:52402) 2023-10-19T10:36:32.275 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 -- 172.21.15.183:0/4237670490 learned_addr learned my addr 172.21.15.183:0/4237670490 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-10-19T10:36:32.275 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcf7fe640 1 --2- 172.21.15.183:0/4237670490 >> v2:172.21.15.50:3300/0 conn(0x7f6bd0160220 0x7f6bd00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.276 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327977564 0 0) 0x7f6bd00b7ff0 con 0x7f6bd00cfa60 2023-10-19T10:36:32.276 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb8003520 con 0x7f6bd00cfa60 2023-10-19T10:36:32.276 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706983980 0 0) 0x7f6bd00d06d0 con 0x7f6bd015c690 2023-10-19T10:36:32.276 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd00b7ff0 con 0x7f6bd015c690 2023-10-19T10:36:32.276 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756384631 0 0) 0x7f6bb8003520 con 0x7f6bd00cfa60 2023-10-19T10:36:32.277 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 >> v2:172.21.15.50:3300/0 conn(0x7f6bd0160220 msgr2=0x7f6bd00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:32.277 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 --2- 172.21.15.183:0/4237670490 >> v2:172.21.15.50:3300/0 conn(0x7f6bd0160220 0x7f6bd00b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.277 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 >> v1:172.21.15.50:6789/0 conn(0x7f6bd015c690 legacy=0x7f6bd00cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.277 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd01609e0 con 0x7f6bd00cfa60 2023-10-19T10:36:32.277 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f6bc40015f0 con 0x7f6bd00cfa60 2023-10-19T10:36:32.277 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 >> v1:172.21.15.183:6789/0 conn(0x7f6bd00cfa60 legacy=0x7f6bd00d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.278 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bb8003b50 0x7f6bb8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.278 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.278 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bb80051d0 0x7f6bb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.278 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0160700 con 0x7f6bb8003b50 2023-10-19T10:36:32.279 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb800c100 con 0x7f6bb80047c0 2023-10-19T10:36:32.279 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb800c2c0 con 0x7f6bb80051d0 2023-10-19T10:36:32.279 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.279 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bcf7fe640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bb80051d0 0x7f6bb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.279 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bceffd640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bb8003b50 0x7f6bb8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.279 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 -- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bb80051d0 msgr2=0x7f6bb800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:32.280 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bb80051d0 0x7f6bb800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.280 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 -- 172.21.15.183:0/4237670490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bb8003b50 msgr2=0x7f6bb8003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:32.280 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bb8003b50 0x7f6bb8003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.280 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.269+0000 7f6bce7fc640 1 -- 172.21.15.183:0/4237670490 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8003520 con 0x7f6bb80047c0 2023-10-19T10:36:32.280 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.270+0000 7f6bcf7fe640 1 -- 172.21.15.183:0/4237670490 reap_dead start 2023-10-19T10:36:32.281 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.270+0000 7f6bce7fc640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bb8004c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4007f10 tx=0x7f6bb80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f9155efbf29042 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.281 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.270+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4005ad0 con 0x7f6bb80047c0 2023-10-19T10:36:32.281 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.270+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc4005c50 con 0x7f6bb80047c0 2023-10-19T10:36:32.281 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.270+0000 7f6bcdffb640 1 -- 172.21.15.183:0/4237670490 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4007070 con 0x7f6bb80047c0 2023-10-19T10:36:32.281 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 -- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 msgr2=0x7f6bb8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.282 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bb8004c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4007f10 tx=0x7f6bb80057f0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.282 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 -- 172.21.15.183:0/4237670490 shutdown_connections 2023-10-19T10:36:32.282 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 --2- 172.21.15.183:0/4237670490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bb8004c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.282 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 -- 172.21.15.183:0/4237670490 >> 172.21.15.183:0/4237670490 conn(0x7f6bd0071110 msgr2=0x7f6bd00714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.282 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 -- 172.21.15.183:0/4237670490 shutdown_connections 2023-10-19T10:36:32.283 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.271+0000 7f6bd5612640 1 -- 172.21.15.183:0/4237670490 wait complete. 2023-10-19T10:36:32.285 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.272+0000 7f6bd5612640 1 Processor -- start 2023-10-19T10:36:32.285 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.272+0000 7f6bd5612640 1 -- start start 2023-10-19T10:36:32.286 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bd5612640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bd00aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.286 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bd5612640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bd014c8d0 0x7f6bd00cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.286 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bd5612640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bd014cd10 0x7f6bd00cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.287 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bd5612640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00cc320 con 0x7f6bd014c8d0 2023-10-19T10:36:32.287 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bd5612640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00aa840 con 0x7f6bb80047c0 2023-10-19T10:36:32.287 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bd5612640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00a9c30 con 0x7f6bd014cd10 2023-10-19T10:36:32.287 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bd00aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.288 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bd014cd10 0x7f6bd00cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.288 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bd00aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39752/0 (socket says 172.21.15.183:39752) 2023-10-19T10:36:32.288 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 -- 172.21.15.183:0/1853598752 learned_addr learned my addr 172.21.15.183:0/1853598752 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:32.288 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bce7fc640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bd014c8d0 0x7f6bd00cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.288 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 -- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bd014cd10 msgr2=0x7f6bd00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.289 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bd014cd10 0x7f6bd00cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.289 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 -- 172.21.15.183:0/1853598752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bd014c8d0 msgr2=0x7f6bd00cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.289 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bd014c8d0 0x7f6bd00cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.289 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bceffd640 1 -- 172.21.15.183:0/1853598752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00a9f10 con 0x7f6bb80047c0 2023-10-19T10:36:32.289 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.273+0000 7f6bcf7fe640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bd014cd10 0x7f6bd00cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:32.290 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bceffd640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bd00aacd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f770 tx=0x7f6bc000fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.290 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc000fe80 con 0x7f6bb80047c0 2023-10-19T10:36:32.290 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc0005320 con 0x7f6bb80047c0 2023-10-19T10:36:32.290 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0006a60 con 0x7f6bb80047c0 2023-10-19T10:36:32.290 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd00aa1f0 con 0x7f6bb80047c0 2023-10-19T10:36:32.291 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd00a8f90 con 0x7f6bb80047c0 2023-10-19T10:36:32.291 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6bc0007050 con 0x7f6bb80047c0 2023-10-19T10:36:32.291 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.274+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f6bc0003640 con 0x7f6bb80047c0 2023-10-19T10:36:32.291 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.275+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd0073990 con 0x7f6bb80047c0 2023-10-19T10:36:32.291 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.277+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6bc001a070 con 0x7f6bb80047c0 2023-10-19T10:36:32.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.347+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2acb3a97-1331-427e-a927-b37554c81873", "id": 6} v 0) v1 -- 0x7f6bd006b700 con 0x7f6bb80047c0 2023-10-19T10:36:32.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.350+0000 7f6bb77fe640 1 -- 172.21.15.183:0/1853598752 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2acb3a97-1331-427e-a927-b37554c81873", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6bc00074e0 con 0x7f6bb80047c0 2023-10-19T10:36:32.354 INFO:teuthology.orchestra.run.smithi183.stdout:6 2023-10-19T10:36:32.356 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.351+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 msgr2=0x7f6bd00aacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.356 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.351+0000 7f6bd5612640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bd00aacd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f770 tx=0x7f6bc000fc30 comp rx=0 tx=0).stop 2023-10-19T10:36:32.356 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 shutdown_connections 2023-10-19T10:36:32.356 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6bd014cd10 0x7f6bd00cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.357 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6bd014c8d0 0x7f6bd00cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.357 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 --2- 172.21.15.183:0/1853598752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb80047c0 0x7f6bd00aacd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.357 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 >> 172.21.15.183:0/1853598752 conn(0x7f6bd0071110 msgr2=0x7f6bd015e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.357 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 shutdown_connections 2023-10-19T10:36:32.357 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.352+0000 7f6bd5612640 1 -- 172.21.15.183:0/1853598752 wait complete. 2023-10-19T10:36:32.365 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 956a9722-f431-4874-b98b-c3be06fb3e14 7 2023-10-19T10:36:32.368 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:32.365+0000 7ff14a232040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-19T10:36:32.466 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.463+0000 7f1ffece7640 1 Processor -- start 2023-10-19T10:36:32.466 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.463+0000 7f1ffece7640 1 -- start start 2023-10-19T10:36:32.468 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ffece7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1ff815e530 0x7f1ff80cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.468 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ffece7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff80b3eb0 con 0x7f1ff80b3740 2023-10-19T10:36:32.468 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ffece7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff80b4190 con 0x7f1ff815a1e0 2023-10-19T10:36:32.468 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ffece7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff80b4ac0 con 0x7f1ff815e530 2023-10-19T10:36:32.468 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ffca83640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f1ff815a1e0 0x7f1ff815de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:52410/0 (socket says 172.21.15.183:52410) 2023-10-19T10:36:32.469 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ffca83640 1 -- 172.21.15.183:0/3435519287 learned_addr learned my addr 172.21.15.183:0/3435519287 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-10-19T10:36:32.469 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ff7fff640 1 --2- 172.21.15.183:0/3435519287 >> v2:172.21.15.50:3300/0 conn(0x7f1ff815e530 0x7f1ff80cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.469 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ff77fe640 1 -- 172.21.15.183:0/3435519287 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921673060 0 0) 0x7f1ff80b4190 con 0x7f1ff815a1e0 2023-10-19T10:36:32.469 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ff77fe640 1 -- 172.21.15.183:0/3435519287 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8003520 con 0x7f1ff815a1e0 2023-10-19T10:36:32.469 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.464+0000 7f1ff77fe640 1 -- 172.21.15.183:0/3435519287 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4237584489 0 0) 0x7f1fe8003520 con 0x7f1ff815a1e0 2023-10-19T10:36:32.470 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.465+0000 7f1ff7fff640 1 -- 172.21.15.183:0/3435519287 >> v1:172.21.15.183:6789/0 conn(0x7f1ff815a1e0 legacy=0x7f1ff815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.470 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.465+0000 7f1ff7fff640 1 -- 172.21.15.183:0/3435519287 >> v1:172.21.15.50:6789/0 conn(0x7f1ff80b3740 legacy=0x7f1ff8069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.470 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.465+0000 7f1ff7fff640 1 -- 172.21.15.183:0/3435519287 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80b4470 con 0x7f1ff815e530 2023-10-19T10:36:32.470 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.465+0000 7f1ff7fff640 1 --2- 172.21.15.183:0/3435519287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff815e530 0x7f1ff80cac50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009720 tx=0x7f1fe402b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb937682ae79d85 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.470 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.465+0000 7f1ff77fe640 1 -- 172.21.15.183:0/3435519287 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec001670 con 0x7f1ff815e530 2023-10-19T10:36:32.471 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ff77fe640 1 -- 172.21.15.183:0/3435519287 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe402e050 con 0x7f1ff815e530 2023-10-19T10:36:32.471 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ff77fe640 1 -- 172.21.15.183:0/3435519287 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4030730 con 0x7f1ff815e530 2023-10-19T10:36:32.474 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 -- 172.21.15.183:0/3435519287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff815e530 msgr2=0x7f1ff80cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.474 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 --2- 172.21.15.183:0/3435519287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff815e530 0x7f1ff80cac50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009720 tx=0x7f1fe402b570 comp rx=0 tx=0).stop 2023-10-19T10:36:32.474 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 -- 172.21.15.183:0/3435519287 shutdown_connections 2023-10-19T10:36:32.474 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 --2- 172.21.15.183:0/3435519287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff815e530 0x7f1ff80cac50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.475 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 -- 172.21.15.183:0/3435519287 >> 172.21.15.183:0/3435519287 conn(0x7f1ff80cb390 msgr2=0x7f1ff80cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.475 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 -- 172.21.15.183:0/3435519287 shutdown_connections 2023-10-19T10:36:32.475 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.466+0000 7f1ffece7640 1 -- 172.21.15.183:0/3435519287 wait complete. 2023-10-19T10:36:32.476 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.467+0000 7f1ffece7640 1 Processor -- start 2023-10-19T10:36:32.476 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.467+0000 7f1ffece7640 1 -- start start 2023-10-19T10:36:32.476 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffece7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff80b3740 0x7f1ff81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.476 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffece7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 0x7f1ff815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.476 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffece7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815e530 0x7f1ff80c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.477 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffece7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff815c400 con 0x7f1ff80b3740 2023-10-19T10:36:32.477 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffece7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80c4270 con 0x7f1ff815a1e0 2023-10-19T10:36:32.477 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffece7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80c4550 con 0x7f1ff815e530 2023-10-19T10:36:32.477 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ff7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 0x7f1ff815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.477 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffd284640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815e530 0x7f1ff80c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.478 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ff7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 0x7f1ff815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39760/0 (socket says 172.21.15.183:39760) 2023-10-19T10:36:32.478 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ff7fff640 1 -- 172.21.15.183:0/1537691185 learned_addr learned my addr 172.21.15.183:0/1537691185 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-10-19T10:36:32.478 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.468+0000 7f1ffca83640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff80b3740 0x7f1ff81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.478 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffd284640 1 -- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 msgr2=0x7f1ff815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.478 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffd284640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 0x7f1ff815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.479 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffd284640 1 -- 172.21.15.183:0/1537691185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff80b3740 msgr2=0x7f1ff81594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.479 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffd284640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff80b3740 0x7f1ff81594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.479 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffd284640 1 -- 172.21.15.183:0/1537691185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4009400 con 0x7f1ff815e530 2023-10-19T10:36:32.479 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ff7fff640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 0x7f1ff815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:32.479 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffd284640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815e530 0x7f1ff80c38b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000c7e0 tx=0x7f1ff000cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.480 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0003da0 con 0x7f1ff815e530 2023-10-19T10:36:32.480 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0005070 con 0x7f1ff815e530 2023-10-19T10:36:32.480 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff815c6a0 con 0x7f1ff815e530 2023-10-19T10:36:32.480 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0005440 con 0x7f1ff815e530 2023-10-19T10:36:32.480 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff815abd0 con 0x7f1ff815e530 2023-10-19T10:36:32.480 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ff000a640 con 0x7f1ff815e530 2023-10-19T10:36:32.481 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.469+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1ff0012400 con 0x7f1ff815e530 2023-10-19T10:36:32.481 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.470+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff80b4700 con 0x7f1ff815e530 2023-10-19T10:36:32.481 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.474+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f1ff0017070 con 0x7f1ff815e530 2023-10-19T10:36:32.506 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.504+0000 7f3589a45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-19T10:36:32.510 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:32.508+0000 7ff14a232040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-19T10:36:32.548 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.545+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "956a9722-f431-4874-b98b-c3be06fb3e14", "id": 7} v 0) v1 -- 0x7f1ff8073990 con 0x7f1ff815e530 2023-10-19T10:36:32.552 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.550+0000 7f1ff57fa640 1 -- 172.21.15.183:0/1537691185 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "956a9722-f431-4874-b98b-c3be06fb3e14", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ff000a8f0 con 0x7f1ff815e530 2023-10-19T10:36:32.553 INFO:teuthology.orchestra.run.smithi183.stdout:7 2023-10-19T10:36:32.554 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815e530 msgr2=0x7f1ff80c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.554 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815e530 0x7f1ff80c38b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000c7e0 tx=0x7f1ff000cca0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 shutdown_connections 2023-10-19T10:36:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815e530 0x7f1ff80c38b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815a1e0 0x7f1ff815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 --2- 172.21.15.183:0/1537691185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff80b3740 0x7f1ff81594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.559 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 >> 172.21.15.183:0/1537691185 conn(0x7f1ff80cb390 msgr2=0x7f1ff80ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.559 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 shutdown_connections 2023-10-19T10:36:32.559 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:36:32.551+0000 7f1ffece7640 1 -- 172.21.15.183:0/1537691185 wait complete. 2023-10-19T10:36:32.564 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-19T10:36:32.565 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-19T10:36:32.568 INFO:tasks.ceph.osd.0:Started 2023-10-19T10:36:32.569 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-19T10:36:32.569 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-19T10:36:32.572 INFO:tasks.ceph.osd.1:Started 2023-10-19T10:36:32.572 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-19T10:36:32.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-19T10:36:32.575 INFO:tasks.ceph.osd.2:Started 2023-10-19T10:36:32.576 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-19T10:36:32.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-19T10:36:32.579 INFO:tasks.ceph.osd.3:Started 2023-10-19T10:36:32.579 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-19T10:36:32.580 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-19T10:36:32.582 INFO:tasks.ceph.osd.4:Started 2023-10-19T10:36:32.583 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-19T10:36:32.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-19T10:36:32.586 INFO:tasks.ceph.osd.5:Started 2023-10-19T10:36:32.587 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-19T10:36:32.587 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-19T10:36:32.590 INFO:tasks.ceph.osd.6:Started 2023-10-19T10:36:32.590 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-19T10:36:32.591 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-19T10:36:32.594 INFO:tasks.ceph.osd.7:Started 2023-10-19T10:36:32.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:36:32.643 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:32.640+0000 7ff14a232040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-19T10:36:32.683 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.681+0000 7f3589a45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-19T10:36:32.702 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:32.700+0000 7ff14a232040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-19T10:36:32.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.728+0000 7efe7f0c4640 1 Processor -- start 2023-10-19T10:36:32.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.728+0000 7efe7f0c4640 1 -- start start 2023-10-19T10:36:32.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7f0c4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7efe780c9790 0x7efe7806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7f0c4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efe780ccd10 0x7efe780cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7f0c4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7806e140 con 0x7efe7815c580 2023-10-19T10:36:32.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7f0c4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7806ec10 con 0x7efe780c9790 2023-10-19T10:36:32.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7f0c4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7815d070 con 0x7efe780ccd10 2023-10-19T10:36:32.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7ce60640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7efe7815c580 0x7efe7815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:35240/0 (socket says 172.21.15.50:35240) 2023-10-19T10:36:32.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe7ce60640 1 -- 172.21.15.50:0/1285156034 learned_addr learned my addr 172.21.15.50:0/1285156034 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:32.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.729+0000 7efe77fff640 1 --2- 172.21.15.50:0/1285156034 >> v2:172.21.15.50:3300/0 conn(0x7efe780c9790 0x7efe7806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe7d661640 1 --2- 172.21.15.50:0/1285156034 >> v2:172.21.15.183:3300/0 conn(0x7efe780ccd10 0x7efe780cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe77fff640 1 -- 172.21.15.50:0/1285156034 >> v2:172.21.15.183:3300/0 conn(0x7efe780ccd10 msgr2=0x7efe780cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:32.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe77fff640 1 --2- 172.21.15.50:0/1285156034 >> v2:172.21.15.183:3300/0 conn(0x7efe780ccd10 0x7efe780cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe77fff640 1 -- 172.21.15.50:0/1285156034 >> v1:172.21.15.50:6789/0 conn(0x7efe7815c580 legacy=0x7efe7815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe77fff640 1 -- 172.21.15.50:0/1285156034 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780709b0 con 0x7efe780c9790 2023-10-19T10:36:32.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe777fe640 1 -- 172.21.15.50:0/1285156034 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184933706 0 0) 0x7efe7806e140 con 0x7efe7815c580 2023-10-19T10:36:32.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.730+0000 7efe77fff640 1 --2- 172.21.15.50:0/1285156034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe780c9790 0x7efe7806dc00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe780c2bf0 tx=0x7efe6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dd0dcef9f8ad01 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe777fe640 1 -- 172.21.15.50:0/1285156034 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64031830 con 0x7efe780c9790 2023-10-19T10:36:32.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe777fe640 1 -- 172.21.15.50:0/1285156034 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64031990 con 0x7efe780c9790 2023-10-19T10:36:32.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe777fe640 1 -- 172.21.15.50:0/1285156034 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64031d20 con 0x7efe780c9790 2023-10-19T10:36:32.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 -- 172.21.15.50:0/1285156034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe780c9790 msgr2=0x7efe7806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/1285156034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe780c9790 0x7efe7806dc00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe780c2bf0 tx=0x7efe6402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 -- 172.21.15.50:0/1285156034 shutdown_connections 2023-10-19T10:36:32.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/1285156034 >> v2:172.21.15.183:3300/0 conn(0x7efe780ccd10 0x7efe780cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/1285156034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe780c9790 0x7efe7806dc00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 -- 172.21.15.50:0/1285156034 >> 172.21.15.50:0/1285156034 conn(0x7efe780cbbc0 msgr2=0x7efe780cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 -- 172.21.15.50:0/1285156034 shutdown_connections 2023-10-19T10:36:32.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.731+0000 7efe7f0c4640 1 -- 172.21.15.50:0/1285156034 wait complete. 2023-10-19T10:36:32.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.733+0000 7efe7f0c4640 1 Processor -- start 2023-10-19T10:36:32.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.733+0000 7efe7f0c4640 1 -- start start 2023-10-19T10:36:32.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.733+0000 7efe7f0c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe780c9790 0x7efe7815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.733+0000 7efe7f0c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe780ccd10 0x7efe7814f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.733+0000 7efe7f0c4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 0x7efe780ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:32.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7f0c4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7815a6b0 con 0x7efe7815c580 2023-10-19T10:36:32.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7f0c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe780acd00 con 0x7efe780c9790 2023-10-19T10:36:32.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7f0c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe780c7510 con 0x7efe780ccd10 2023-10-19T10:36:32.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 0x7efe780ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7ce60640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe780c9790 0x7efe7815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 0x7efe780ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46690/0 (socket says 172.21.15.50:46690) 2023-10-19T10:36:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 -- 172.21.15.50:0/2927953233 learned_addr learned my addr 172.21.15.50:0/2927953233 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe77fff640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe780ccd10 0x7efe7814f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 -- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe780ccd10 msgr2=0x7efe7814f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe780ccd10 0x7efe7814f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 -- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe780c9790 msgr2=0x7efe7815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe780c9790 0x7efe7815a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 -- 172.21.15.50:0/2927953233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe64009400 con 0x7efe7815c580 2023-10-19T10:36:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7ce60640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe780c9790 0x7efe7815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7d661640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 0x7efe780ad0a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe70007e70 tx=0x7efe7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70014620 con 0x7efe7815c580 2023-10-19T10:36:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700147a0 con 0x7efe7815c580 2023-10-19T10:36:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.734+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe780ac740 con 0x7efe7815c580 2023-10-19T10:36:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.735+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70014b50 con 0x7efe7815c580 2023-10-19T10:36:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.735+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe780c7790 con 0x7efe7815c580 2023-10-19T10:36:32.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.735+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efe7001e460 con 0x7efe7815c580 2023-10-19T10:36:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.735+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7efe7001d600 con 0x7efe7815c580 2023-10-19T10:36:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.736+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78073990 con 0x7efe7815c580 2023-10-19T10:36:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.738+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7efe7001e710 con 0x7efe7815c580 2023-10-19T10:36:32.800 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:32.798+0000 7ff14a232040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-19T10:36:32.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.800+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe780c94d0 con 0x7efe7815c580 2023-10-19T10:36:32.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.800+0000 7efe757fa640 1 -- 172.21.15.50:0/2927953233 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7efe7002d090 con 0x7efe7815c580 2023-10-19T10:36:32.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:36:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":10,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:32.547423+0000","last_up_change":"0.000000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:36:32.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 msgr2=0x7efe780ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 0x7efe780ad0a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe70007e70 tx=0x7efe7000e4b0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 shutdown_connections 2023-10-19T10:36:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe7815c580 0x7efe780ad0a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe780ccd10 0x7efe7814f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 --2- 172.21.15.50:0/2927953233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe780c9790 0x7efe7815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 >> 172.21.15.50:0/2927953233 conn(0x7efe780cbbc0 msgr2=0x7efe78069800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 shutdown_connections 2023-10-19T10:36:32.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:32.801+0000 7efe7f0c4640 1 -- 172.21.15.50:0/2927953233 wait complete. 2023-10-19T10:36:32.814 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-19T10:36:32.815 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-19T10:36:32.840 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.837+0000 7f3589a45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-19T10:36:32.888 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.885+0000 7f3589a45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-19T10:36:32.895 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:32.893+0000 7ff14a232040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-19T10:36:32.989 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:32.987+0000 7f3589a45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-19T10:36:33.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:33.033+0000 7ff14a232040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-19T10:36:33.078 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:33.075+0000 7ff14a232040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-19T10:36:33.088 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:33.085+0000 7f3589a45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-19T10:36:33.186 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:36:33.183+0000 7fd5c1618500 -1 Falling back to public interface 2023-10-19T10:36:33.190 INFO:tasks.ceph.osd.3.smithi050.stderr:2023-10-19T10:36:33.187+0000 7f18bf218500 -1 Falling back to public interface 2023-10-19T10:36:33.190 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:36:33.188+0000 7f8c66cb6500 -1 Falling back to public interface 2023-10-19T10:36:33.191 INFO:tasks.ceph.osd.0.smithi050.stderr:2023-10-19T10:36:33.189+0000 7efc3f0af500 -1 Falling back to public interface 2023-10-19T10:36:33.192 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-10-19T10:36:33.189+0000 7efc130cf500 -1 Falling back to public interface 2023-10-19T10:36:33.192 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:36:33.189+0000 7f8c65c5d640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6813/0: (98) Address already in use 2023-10-19T10:36:33.198 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:36:33.196+0000 7f1b49a1a500 -1 Falling back to public interface 2023-10-19T10:36:33.213 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:36:33.211+0000 7f5c7b810500 -1 Falling back to public interface 2023-10-19T10:36:33.217 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:33.215+0000 7ff14a232040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-19T10:36:33.220 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:36:33.218+0000 7fbcddbe2500 -1 Falling back to public interface 2023-10-19T10:36:33.237 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:33.235+0000 7f3589a45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-19T10:36:33.281 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:33.279+0000 7f3589a45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-19T10:36:33.428 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:33.426+0000 7f3589a45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-19T10:36:33.620 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-19T10:36:33.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.709+0000 7ff240a4d640 1 Processor -- start 2023-10-19T10:36:33.712 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.709+0000 7ff240a4d640 1 -- start start 2023-10-19T10:36:33.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff240a4d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff23c0ccd60 0x7ff23c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:33.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff240a4d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c160520 con 0x7ff23c06d460 2023-10-19T10:36:33.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff240a4d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c1610c0 con 0x7ff23c0cfa60 2023-10-19T10:36:33.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff240a4d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c15d2a0 con 0x7ff23c0ccd60 2023-10-19T10:36:33.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff233fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff23c06d460 0x7ff23c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:35242/0 (socket says 172.21.15.50:35242) 2023-10-19T10:36:33.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff233fff640 1 -- 172.21.15.50:0/3958059923 learned_addr learned my addr 172.21.15.50:0/3958059923 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:33.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff23ad9d640 1 --2- 172.21.15.50:0/3958059923 >> v2:172.21.15.183:3300/0 conn(0x7ff23c0ccd60 0x7ff23c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:33.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff239d9b640 1 -- 172.21.15.50:0/3958059923 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483730486 0 0) 0x7ff23c160520 con 0x7ff23c06d460 2023-10-19T10:36:33.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff239d9b640 1 -- 172.21.15.50:0/3958059923 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff228003520 con 0x7ff23c06d460 2023-10-19T10:36:33.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff239d9b640 1 -- 172.21.15.50:0/3958059923 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399762167 0 0) 0x7ff228003520 con 0x7ff23c06d460 2023-10-19T10:36:33.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff23ad9d640 1 -- 172.21.15.50:0/3958059923 >> v1:172.21.15.183:6789/0 conn(0x7ff23c0cfa60 legacy=0x7ff23c15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:33.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff23ad9d640 1 -- 172.21.15.50:0/3958059923 >> v1:172.21.15.50:6789/0 conn(0x7ff23c06d460 legacy=0x7ff23c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:33.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.710+0000 7ff23ad9d640 1 -- 172.21.15.50:0/3958059923 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c160220 con 0x7ff23c0ccd60 2023-10-19T10:36:33.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.711+0000 7ff23ad9d640 1 --2- 172.21.15.50:0/3958059923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c0cd180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff234009d90 tx=0x7ff23402bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4318eeb1275add96 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:33.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.711+0000 7ff239d9b640 1 -- 172.21.15.50:0/3958059923 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224001670 con 0x7ff23c0ccd60 2023-10-19T10:36:33.717 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.711+0000 7ff239d9b640 1 -- 172.21.15.50:0/3958059923 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23402c8b0 con 0x7ff23c0ccd60 2023-10-19T10:36:33.717 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.711+0000 7ff239d9b640 1 -- 172.21.15.50:0/3958059923 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23402cc40 con 0x7ff23c0ccd60 2023-10-19T10:36:33.719 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 -- 172.21.15.50:0/3958059923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 msgr2=0x7ff23c0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:33.720 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 --2- 172.21.15.50:0/3958059923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c0cd180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff234009d90 tx=0x7ff23402bd80 comp rx=0 tx=0).stop 2023-10-19T10:36:33.720 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 -- 172.21.15.50:0/3958059923 shutdown_connections 2023-10-19T10:36:33.720 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 --2- 172.21.15.50:0/3958059923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c0cd180 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.721 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 -- 172.21.15.50:0/3958059923 >> 172.21.15.50:0/3958059923 conn(0x7ff23c06ad00 msgr2=0x7ff23c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:33.721 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 -- 172.21.15.50:0/3958059923 shutdown_connections 2023-10-19T10:36:33.721 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.712+0000 7ff240a4d640 1 -- 172.21.15.50:0/3958059923 wait complete. 2023-10-19T10:36:33.721 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.713+0000 7ff240a4d640 1 Processor -- start 2023-10-19T10:36:33.721 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.713+0000 7ff240a4d640 1 -- start start 2023-10-19T10:36:33.722 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff240a4d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 0x7ff23c0c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:33.722 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff240a4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:33.722 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff240a4d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff23c0cfa60 0x7ff23c0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:33.722 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff240a4d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c1f6380 con 0x7ff23c06d460 2023-10-19T10:36:33.722 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff240a4d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0c5ac0 con 0x7ff23c0ccd60 2023-10-19T10:36:33.723 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff240a4d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0c5500 con 0x7ff23c0cfa60 2023-10-19T10:36:33.723 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 0x7ff23c0c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:33.723 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 0x7ff23c0c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46700/0 (socket says 172.21.15.50:46700) 2023-10-19T10:36:33.723 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 -- 172.21.15.50:0/2258258530 learned_addr learned my addr 172.21.15.50:0/2258258530 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:33.723 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff233fff640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:33.724 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23ad9d640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff23c0cfa60 0x7ff23c0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:33.724 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 -- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff23c0cfa60 msgr2=0x7ff23c0c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:33.724 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff23c0cfa60 0x7ff23c0c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.724 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 -- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 msgr2=0x7ff23c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:33.724 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.724 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff23a59c640 1 -- 172.21.15.50:0/2258258530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234009a70 con 0x7ff23c06d460 2023-10-19T10:36:33.725 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.714+0000 7ff233fff640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:33.725 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff23a59c640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 0x7ff23c0c4f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c400 tx=0x7ff22c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:33.725 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0052d0 con 0x7ff23c06d460 2023-10-19T10:36:33.725 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c005430 con 0x7ff23c06d460 2023-10-19T10:36:33.725 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0c57e0 con 0x7ff23c06d460 2023-10-19T10:36:33.726 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c003830 con 0x7ff23c06d460 2023-10-19T10:36:33.726 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c0c3580 con 0x7ff23c06d460 2023-10-19T10:36:33.726 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff22c003c90 con 0x7ff23c06d460 2023-10-19T10:36:33.726 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.715+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff22c012af0 con 0x7ff23c06d460 2023-10-19T10:36:33.726 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.716+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c067330 con 0x7ff23c06d460 2023-10-19T10:36:33.727 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.718+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff22c017070 con 0x7ff23c06d460 2023-10-19T10:36:33.782 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.780+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff23c0c37a0 con 0x7ff23c06d460 2023-10-19T10:36:33.785 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff232ffd640 1 -- 172.21.15.50:0/2258258530 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff22c012420 con 0x7ff23c06d460 2023-10-19T10:36:33.786 INFO:teuthology.misc.health.smithi050.stdout: 2023-10-19T10:36:33.786 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":10,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:32.547423+0000","last_up_change":"0.000000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:36:33.787 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 msgr2=0x7ff23c0c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:33.787 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff240a4d640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 0x7ff23c0c4f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c400 tx=0x7ff22c00c8c0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.788 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 shutdown_connections 2023-10-19T10:36:33.788 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff240a4d640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff23c0cfa60 0x7ff23c0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.788 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff240a4d640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23c0ccd60 0x7ff23c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.788 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.781+0000 7ff240a4d640 1 --2- 172.21.15.50:0/2258258530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c06d460 0x7ff23c0c4f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:33.788 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.782+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 >> 172.21.15.50:0/2258258530 conn(0x7ff23c06ad00 msgr2=0x7ff23c072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:33.789 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.782+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 shutdown_connections 2023-10-19T10:36:33.789 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:33.782+0000 7ff240a4d640 1 -- 172.21.15.50:0/2258258530 wait complete. 2023-10-19T10:36:33.793 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-19T10:36:34.018 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:34.014+0000 7ff14a232040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-19T10:36:34.266 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:34.264+0000 7f3589a45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-19T10:36:34.286 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:34.284+0000 7ff14a232040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-19T10:36:34.339 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:34.336+0000 7ff14a232040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-19T10:36:34.439 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:34.436+0000 7ff14a232040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-19T10:36:34.555 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:34.552+0000 7f3589a45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-19T10:36:34.610 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:34.606+0000 7f3589a45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-19T10:36:34.713 INFO:tasks.ceph.mgr.y.smithi050.stderr:2023-10-19T10:36:34.710+0000 7f3589a45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-19T10:36:35.466 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:35.463+0000 7ff0eaefc640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-19T10:36:35.466 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-19T10:36:35.464+0000 7ff0eaefc640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-19T10:36:36.654 INFO:tasks.ceph.osd.3.smithi050.stderr:2023-10-19T10:36:36.652+0000 7f18bf218500 -1 osd.3 0 log_to_monitors true 2023-10-19T10:36:36.677 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:36:36.674+0000 7f8c66cb6500 -1 osd.2 0 log_to_monitors true 2023-10-19T10:36:36.689 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:36:36.687+0000 7f1b49a1a500 -1 osd.1 0 log_to_monitors true 2023-10-19T10:36:36.716 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-10-19T10:36:36.713+0000 7efc130cf500 -1 osd.5 0 log_to_monitors true 2023-10-19T10:36:36.717 INFO:tasks.ceph.osd.0.smithi050.stderr:2023-10-19T10:36:36.714+0000 7efc3f0af500 -1 osd.0 0 log_to_monitors true 2023-10-19T10:36:36.731 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:36:36.728+0000 7f5c7b810500 -1 osd.6 0 log_to_monitors true 2023-10-19T10:36:36.734 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:36:36.731+0000 7fd5c1618500 -1 osd.4 0 log_to_monitors true 2023-10-19T10:36:36.739 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:36:36.736+0000 7fbcddbe2500 -1 osd.7 0 log_to_monitors true 2023-10-19T10:36:39.893 INFO:tasks.ceph.osd.3.smithi050.stderr:2023-10-19T10:36:39.891+0000 7f18bac74640 -1 osd.3 0 waiting for initial osdmap 2023-10-19T10:36:39.909 INFO:tasks.ceph.osd.3.smithi050.stderr:2023-10-19T10:36:39.907+0000 7f18b2a95640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:39.910 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:36:39.907+0000 7f8c62c42640 -1 osd.2 0 waiting for initial osdmap 2023-10-19T10:36:39.932 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:36:39.930+0000 7f8c5aa63640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:39.934 INFO:tasks.ceph.osd.0.smithi050.stderr:2023-10-19T10:36:39.932+0000 7efc3b03b640 -1 osd.0 0 waiting for initial osdmap 2023-10-19T10:36:39.951 INFO:tasks.ceph.osd.0.smithi050.stderr:2023-10-19T10:36:39.949+0000 7efc32e5c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:39.996 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:36:39.994+0000 7f1b4547b640 -1 osd.1 0 waiting for initial osdmap 2023-10-19T10:36:40.011 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:36:40.009+0000 7f1b3d29c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:40.361 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-10-19T10:36:40.357+0000 7efc0f05b640 -1 osd.5 0 waiting for initial osdmap 2023-10-19T10:36:40.392 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-10-19T10:36:40.389+0000 7efc06e7c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:40.492 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:36:40.489+0000 7fd5bd003640 -1 osd.4 0 waiting for initial osdmap 2023-10-19T10:36:40.508 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:36:40.505+0000 7f5c7728d640 -1 osd.6 0 waiting for initial osdmap 2023-10-19T10:36:40.533 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:36:40.530+0000 7fd5b4e24640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:40.546 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:36:40.543+0000 7f5c6f0ae640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:40.600 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-19T10:36:40.699 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.697+0000 7ffa701a0640 1 Processor -- start 2023-10-19T10:36:40.700 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.697+0000 7ffa701a0640 1 -- start start 2023-10-19T10:36:40.701 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa701a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa680afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:40.701 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa701a0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa680cdb10 con 0x7ffa680af200 2023-10-19T10:36:40.702 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa701a0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa680cddf0 con 0x7ffa680d0530 2023-10-19T10:36:40.702 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa701a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680ce0d0 con 0x7ffa680cf970 2023-10-19T10:36:40.702 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6d73b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ffa680af200 0x7ffa680af6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:37120/0 (socket says 172.21.15.50:37120) 2023-10-19T10:36:40.702 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6d73b640 1 -- 172.21.15.50:0/1518882741 learned_addr learned my addr 172.21.15.50:0/1518882741 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:40.702 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6e73d640 1 --2- 172.21.15.50:0/1518882741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa680afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:40.703 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954069668 0 0) 0x7ffa680cdb10 con 0x7ffa680af200 2023-10-19T10:36:40.703 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa54003520 con 0x7ffa680af200 2023-10-19T10:36:40.703 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597326530 0 0) 0x7ffa680cddf0 con 0x7ffa680d0530 2023-10-19T10:36:40.703 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa680cdb10 con 0x7ffa680d0530 2023-10-19T10:36:40.703 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6e73d640 1 -- 172.21.15.50:0/1518882741 >> v1:172.21.15.183:6789/0 conn(0x7ffa680d0530 legacy=0x7ffa680d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.704 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6e73d640 1 -- 172.21.15.50:0/1518882741 >> v1:172.21.15.50:6789/0 conn(0x7ffa680af200 legacy=0x7ffa680af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.704 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.698+0000 7ffa6e73d640 1 -- 172.21.15.50:0/1518882741 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa680ce3b0 con 0x7ffa680cf970 2023-10-19T10:36:40.707 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.699+0000 7ffa6e73d640 1 --2- 172.21.15.50:0/1518882741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa680afdc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009dc0 tx=0x7ffa6002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793b8b03835abe1c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:40.708 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.699+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6002cc40 con 0x7ffa680cf970 2023-10-19T10:36:40.708 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.699+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6002cda0 con 0x7ffa680cf970 2023-10-19T10:36:40.708 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.699+0000 7ffa6cf3a640 1 -- 172.21.15.50:0/1518882741 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60034640 con 0x7ffa680cf970 2023-10-19T10:36:40.708 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 -- 172.21.15.50:0/1518882741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 msgr2=0x7ffa680afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.709 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 --2- 172.21.15.50:0/1518882741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa680afdc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009dc0 tx=0x7ffa6002be90 comp rx=0 tx=0).stop 2023-10-19T10:36:40.709 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 -- 172.21.15.50:0/1518882741 shutdown_connections 2023-10-19T10:36:40.709 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 --2- 172.21.15.50:0/1518882741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa680afdc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.709 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 -- 172.21.15.50:0/1518882741 >> 172.21.15.50:0/1518882741 conn(0x7ffa680d23c0 msgr2=0x7ffa680d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:40.709 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 -- 172.21.15.50:0/1518882741 shutdown_connections 2023-10-19T10:36:40.710 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.700+0000 7ffa701a0640 1 -- 172.21.15.50:0/1518882741 wait complete. 2023-10-19T10:36:40.710 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.701+0000 7ffa701a0640 1 Processor -- start 2023-10-19T10:36:40.710 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.701+0000 7ffa701a0640 1 -- start start 2023-10-19T10:36:40.710 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa701a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 0x7ffa6815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:40.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa701a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa68153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:40.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa701a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa680d0530 0x7ffa680c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:40.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa701a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68153cd0 con 0x7ffa680af200 2023-10-19T10:36:40.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa701a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68154fc0 con 0x7ffa680d0530 2023-10-19T10:36:40.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa701a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6815a610 con 0x7ffa680cf970 2023-10-19T10:36:40.711 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 0x7ffa6815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:40.712 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 0x7ffa6815a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49852/0 (socket says 172.21.15.50:49852) 2023-10-19T10:36:40.712 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 -- 172.21.15.50:0/2816007550 learned_addr learned my addr 172.21.15.50:0/2816007550 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:40.712 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6d73b640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa68153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:40.712 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6e73d640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa680d0530 0x7ffa680c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:40.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 -- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 msgr2=0x7ffa68153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa68153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 -- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa680d0530 msgr2=0x7ffa680c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa680d0530 0x7ffa680c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.702+0000 7ffa6df3c640 1 -- 172.21.15.50:0/2816007550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009a70 con 0x7ffa680af200 2023-10-19T10:36:40.713 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.703+0000 7ffa6df3c640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 0x7ffa6815a0d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c430 tx=0x7ffa6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:40.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.703+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa640052d0 con 0x7ffa680af200 2023-10-19T10:36:40.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.703+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64005450 con 0x7ffa680af200 2023-10-19T10:36:40.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.703+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003830 con 0x7ffa680af200 2023-10-19T10:36:40.714 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.703+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa68153f20 con 0x7ffa680af200 2023-10-19T10:36:40.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.703+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa681544e0 con 0x7ffa680af200 2023-10-19T10:36:40.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.704+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6804e420 con 0x7ffa680af200 2023-10-19T10:36:40.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.704+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7ffa64003990 con 0x7ffa680af200 2023-10-19T10:36:40.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.705+0000 7ffa5e7fc640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffa4c069fc0 0x7ffa4c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:40.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.705+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7ffa64086a30 con 0x7ffa680af200 2023-10-19T10:36:40.715 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.705+0000 7ffa6d73b640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffa4c069fc0 0x7ffa4c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:40.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.705+0000 7ffa6d73b640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffa4c069fc0 0x7ffa4c06c480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa681f51d0 tx=0x7ffa58007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:40.716 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.709+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ffa64053230 con 0x7ffa680af200 2023-10-19T10:36:40.888 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.886+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa680c5c80 con 0x7ffa680af200 2023-10-19T10:36:40.889 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.886+0000 7ffa5e7fc640 1 -- 172.21.15.50:0/2816007550 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ffa64017090 con 0x7ffa680af200 2023-10-19T10:36:40.889 INFO:teuthology.misc.health.smithi050.stdout: 2023-10-19T10:36:40.889 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":12,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:38.466080+0000","last_up_change":"0.000000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:36:40.891 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffa4c069fc0 msgr2=0x7ffa4c06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.891 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffa4c069fc0 0x7ffa4c06c480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa681f51d0 tx=0x7ffa58007d30 comp rx=0 tx=0).stop 2023-10-19T10:36:40.892 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 msgr2=0x7ffa6815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:40.892 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 0x7ffa6815a0d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c430 tx=0x7ffa6400c8f0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.892 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 shutdown_connections 2023-10-19T10:36:40.892 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffa4c069fc0 0x7ffa4c06c480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.893 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa680d0530 0x7ffa680c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.893 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa680cf970 0x7ffa68153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.893 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 --2- 172.21.15.50:0/2816007550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa680af200 0x7ffa6815a0d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:40.893 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 >> 172.21.15.50:0/2816007550 conn(0x7ffa680d23c0 msgr2=0x7ffa681558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:40.894 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 shutdown_connections 2023-10-19T10:36:40.894 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:40.888+0000 7ffa701a0640 1 -- 172.21.15.50:0/2816007550 wait complete. 2023-10-19T10:36:40.900 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-19T10:36:41.843 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:36:41.841+0000 7fbcd9b70640 -1 osd.7 0 waiting for initial osdmap 2023-10-19T10:36:41.855 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:36:41.853+0000 7fbcd1991640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:36:47.707 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-19T10:36:47.815 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.808+0000 7f938fbab640 1 Processor -- start 2023-10-19T10:36:47.815 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.808+0000 7f938fbab640 1 -- start start 2023-10-19T10:36:47.815 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938fbab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9388089950 0x7f93880cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.816 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938fbab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f938806b6b0 0x7f938806bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.816 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938fbab640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93880ccdc0 con 0x7f93880c85b0 2023-10-19T10:36:47.816 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938fbab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93880cde10 con 0x7f938806b6b0 2023-10-19T10:36:47.816 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938fbab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93880cc000 con 0x7f9388089950 2023-10-19T10:36:47.816 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938d947640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f93880c85b0 0x7f93880c8990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48834/0 (socket says 172.21.15.50:48834) 2023-10-19T10:36:47.817 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938d947640 1 -- 172.21.15.50:0/646863297 learned_addr learned my addr 172.21.15.50:0/646863297 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:47.817 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938d146640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9388089950 0x7f93880cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.817 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938e148640 1 --2- 172.21.15.50:0/646863297 >> v2:172.21.15.183:3300/0 conn(0x7f938806b6b0 0x7f938806bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.817 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.809+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74009484 0 0) 0x7f93880ccdc0 con 0x7f93880c85b0 2023-10-19T10:36:47.818 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9370003500 con 0x7f93880c85b0 2023-10-19T10:36:47.818 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805101132 0 0) 0x7f9370003500 con 0x7f93880c85b0 2023-10-19T10:36:47.818 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9388089950 msgr2=0x7f93880cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:47.818 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9388089950 0x7f93880cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.818 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 >> v2:172.21.15.183:3300/0 conn(0x7f938806b6b0 msgr2=0x7f938806bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:47.819 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 --2- 172.21.15.50:0/646863297 >> v2:172.21.15.183:3300/0 conn(0x7f938806b6b0 0x7f938806bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.819 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880cc2e0 con 0x7f93880c85b0 2023-10-19T10:36:47.819 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f9380001670 con 0x7f93880c85b0 2023-10-19T10:36:47.819 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 >> v1:172.21.15.50:6789/0 conn(0x7f93880c85b0 legacy=0x7f93880c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:47.820 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93700060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.820 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9370006600 0x7f9370008a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.820 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370009050 0x7f937000b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.820 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880cc000 con 0x7f9370003cd0 2023-10-19T10:36:47.820 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937000bdc0 con 0x7f9370009050 2023-10-19T10:36:47.821 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937000bf80 con 0x7f9370006600 2023-10-19T10:36:47.821 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938e148640 1 --2- 172.21.15.50:0/646863297 >> v2:172.21.15.183:3300/0 conn(0x7f938806b6b0 0x7f938806bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:47.821 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938d146640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9388089950 0x7f93880cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:47.821 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938d947640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93700060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.821 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.810+0000 7f938e148640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370009050 0x7f937000b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.821 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d146640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9370006600 0x7f9370008a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.822 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d947640 1 -- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9370006600 msgr2=0x7f9370008a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:47.822 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d947640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9370006600 0x7f9370008a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.822 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d947640 1 -- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370009050 msgr2=0x7f937000b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:47.822 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d947640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370009050 0x7f937000b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.822 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d947640 1 -- 172.21.15.50:0/646863297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370003500 con 0x7f9370003cd0 2023-10-19T10:36:47.823 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d146640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9370006600 0x7f9370008a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:47.823 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.811+0000 7f938d947640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93700060c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f938000c240 tx=0x7f93700037b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783489c8a1b372a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:47.823 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938002cb00 con 0x7f9370003cd0 2023-10-19T10:36:47.823 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938002cc60 con 0x7f9370003cd0 2023-10-19T10:36:47.823 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938c945640 1 -- 172.21.15.50:0/646863297 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380005af0 con 0x7f9370003cd0 2023-10-19T10:36:47.824 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938fbab640 1 -- 172.21.15.50:0/646863297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 msgr2=0x7f93700060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:47.824 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938fbab640 1 --2- 172.21.15.50:0/646863297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93700060c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f938000c240 tx=0x7f93700037b0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.824 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938e148640 1 -- 172.21.15.50:0/646863297 reap_dead start 2023-10-19T10:36:47.824 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938fbab640 1 -- 172.21.15.50:0/646863297 shutdown_connections 2023-10-19T10:36:47.824 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938fbab640 1 -- 172.21.15.50:0/646863297 >> 172.21.15.50:0/646863297 conn(0x7f93880afa90 msgr2=0x7f93880afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:47.825 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938fbab640 1 -- 172.21.15.50:0/646863297 shutdown_connections 2023-10-19T10:36:47.825 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.812+0000 7f938fbab640 1 -- 172.21.15.50:0/646863297 wait complete. 2023-10-19T10:36:47.827 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.813+0000 7f938fbab640 1 Processor -- start 2023-10-19T10:36:47.828 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.813+0000 7f938fbab640 1 -- start start 2023-10-19T10:36:47.828 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938fbab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93880c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.828 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938fbab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93881ff9d0 0x7f93881fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.828 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938fbab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881fef00 0x7f93880c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.828 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938fbab640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880c80c0 con 0x7f9370003cd0 2023-10-19T10:36:47.829 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938fbab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881ff410 con 0x7f93881fef00 2023-10-19T10:36:47.829 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938fbab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93881ff6f0 con 0x7f93881ff9d0 2023-10-19T10:36:47.829 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938d947640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93880c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.829 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938d947640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93880c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59936/0 (socket says 172.21.15.50:59936) 2023-10-19T10:36:47.830 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.814+0000 7f938d947640 1 -- 172.21.15.50:0/1136908147 learned_addr learned my addr 172.21.15.50:0/1136908147 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:47.830 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d146640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93881ff9d0 0x7f93881fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.830 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938e148640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881fef00 0x7f93880c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.830 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d947640 1 -- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93881ff9d0 msgr2=0x7f93881fe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:47.830 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d947640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93881ff9d0 0x7f93881fe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.831 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d947640 1 -- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881fef00 msgr2=0x7f93880c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:47.831 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d947640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881fef00 0x7f93880c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:47.831 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d947640 1 -- 172.21.15.50:0/1136908147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93881f3d80 con 0x7f9370003cd0 2023-10-19T10:36:47.831 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938e148640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881fef00 0x7f93880c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:47.831 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938d947640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93880c7b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f938000c240 tx=0x7f9380001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:47.832 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380039030 con 0x7f9370003cd0 2023-10-19T10:36:47.832 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9380004bf0 con 0x7f9370003cd0 2023-10-19T10:36:47.832 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93881f4060 con 0x7f9370003cd0 2023-10-19T10:36:47.832 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938003c040 con 0x7f9370003cd0 2023-10-19T10:36:47.832 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.815+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9388089620 con 0x7f9370003cd0 2023-10-19T10:36:47.833 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.816+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93881f41f0 con 0x7f9370003cd0 2023-10-19T10:36:47.833 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.817+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9380032070 con 0x7f9370003cd0 2023-10-19T10:36:47.833 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.817+0000 7f93867fc640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937406a080 0x7f937406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:47.833 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.817+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93800a9510 con 0x7f9370003cd0 2023-10-19T10:36:47.833 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.817+0000 7f938d146640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937406a080 0x7f937406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:47.834 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.818+0000 7f938d146640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937406a080 0x7f937406c540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9388071680 tx=0x7f937c00f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:47.834 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:47.822+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f93800750b0 con 0x7f9370003cd0 2023-10-19T10:36:48.002 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.000+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93880cc590 con 0x7f9370003cd0 2023-10-19T10:36:48.003 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.001+0000 7f93867fc640 1 -- 172.21.15.50:0/1136908147 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12814 (secure 0 0 0) 0x7f93800786d0 con 0x7f9370003cd0 2023-10-19T10:36:48.004 INFO:teuthology.misc.health.smithi050.stdout: 2023-10-19T10:36:48.004 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":16,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:44.524398+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:36:48.005 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937406a080 msgr2=0x7f937406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:48.006 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937406a080 0x7f937406c540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9388071680 tx=0x7f937c00f040 comp rx=0 tx=0).stop 2023-10-19T10:36:48.006 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 msgr2=0x7f93880c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:48.006 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93880c7b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f938000c240 tx=0x7f9380001a00 comp rx=0 tx=0).stop 2023-10-19T10:36:48.006 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 shutdown_connections 2023-10-19T10:36:48.007 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937406a080 0x7f937406c540 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.007 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881fef00 0x7f93880c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.007 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93881ff9d0 0x7f93881fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.007 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 --2- 172.21.15.50:0/1136908147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9370003cd0 0x7f93880c7b80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.007 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 >> 172.21.15.50:0/1136908147 conn(0x7f93880afa90 msgr2=0x7f938806f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:48.008 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 shutdown_connections 2023-10-19T10:36:48.008 INFO:teuthology.misc.health.smithi050.stderr:2023-10-19T10:36:48.003+0000 7f938fbab640 1 -- 172.21.15.50:0/1136908147 wait complete. 2023-10-19T10:36:48.015 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-19T10:36:48.015 INFO:tasks.ceph:Creating RBD pool 2023-10-19T10:36:48.015 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-19T10:36:48.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.118+0000 7f4ca587e640 1 Processor -- start 2023-10-19T10:36:48.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.118+0000 7f4ca587e640 1 -- start start 2023-10-19T10:36:48.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4ca587e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4ca015c660 0x7f4ca006a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:48.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4ca587e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ca0160240 0x7f4ca00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:48.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4ca587e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca0070780 con 0x7f4ca00c97d0 2023-10-19T10:36:48.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4ca587e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca00cca10 con 0x7f4ca0160240 2023-10-19T10:36:48.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4ca587e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca006a750 con 0x7f4ca015c660 2023-10-19T10:36:48.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4c9e7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ca0160240 0x7f4ca00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:48.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4c9e7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ca0160240 0x7f4ca00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59942/0 (socket says 172.21.15.50:59942) 2023-10-19T10:36:48.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4c9e7fc640 1 -- 172.21.15.50:0/440504525 learned_addr learned my addr 172.21.15.50:0/440504525 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:48.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.119+0000 7f4c9effd640 1 --2- 172.21.15.50:0/440504525 >> v2:172.21.15.183:3300/0 conn(0x7f4ca015c660 0x7f4ca006a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:48.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.120+0000 7f4c9e7fc640 1 -- 172.21.15.50:0/440504525 >> v2:172.21.15.183:3300/0 conn(0x7f4ca015c660 msgr2=0x7f4ca006a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:48.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.120+0000 7f4c9e7fc640 1 --2- 172.21.15.50:0/440504525 >> v2:172.21.15.183:3300/0 conn(0x7f4ca015c660 0x7f4ca006a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.120+0000 7f4c9e7fc640 1 -- 172.21.15.50:0/440504525 >> v1:172.21.15.183:6789/0 conn(0x7f4ca00c97d0 legacy=0x7f4ca006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:48.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.120+0000 7f4c9e7fc640 1 -- 172.21.15.50:0/440504525 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0160720 con 0x7f4ca0160240 2023-10-19T10:36:48.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.120+0000 7f4c9e7fc640 1 --2- 172.21.15.50:0/440504525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00ccd50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00703f0 tx=0x7f4c8802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809773be2c03fe5e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:48.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4c9dffb640 1 -- 172.21.15.50:0/440504525 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88030860 con 0x7f4ca0160240 2023-10-19T10:36:48.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4c9dffb640 1 -- 172.21.15.50:0/440504525 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c880309c0 con 0x7f4ca0160240 2023-10-19T10:36:48.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4c9dffb640 1 -- 172.21.15.50:0/440504525 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88030d50 con 0x7f4ca0160240 2023-10-19T10:36:48.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 -- 172.21.15.50:0/440504525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 msgr2=0x7f4ca00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:48.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 --2- 172.21.15.50:0/440504525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00ccd50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00703f0 tx=0x7f4c8802b9a0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 -- 172.21.15.50:0/440504525 shutdown_connections 2023-10-19T10:36:48.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 --2- 172.21.15.50:0/440504525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00ccd50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 --2- 172.21.15.50:0/440504525 >> v2:172.21.15.183:3300/0 conn(0x7f4ca015c660 0x7f4ca006a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 -- 172.21.15.50:0/440504525 >> 172.21.15.50:0/440504525 conn(0x7f4ca00cbc00 msgr2=0x7f4ca00cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 -- 172.21.15.50:0/440504525 shutdown_connections 2023-10-19T10:36:48.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.121+0000 7f4ca587e640 1 -- 172.21.15.50:0/440504525 wait complete. 2023-10-19T10:36:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.122+0000 7f4ca587e640 1 Processor -- start 2023-10-19T10:36:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 -- start start 2023-10-19T10:36:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ca00c97d0 0x7f4ca0070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ca015c660 0x7f4ca00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:48.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00c6250 con 0x7f4ca0160240 2023-10-19T10:36:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00c8fd0 con 0x7f4ca00c97d0 2023-10-19T10:36:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4ca587e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00c6460 con 0x7f4ca015c660 2023-10-19T10:36:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59952/0 (socket says 172.21.15.50:59952) 2023-10-19T10:36:48.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4c9f7fe640 1 -- 172.21.15.50:0/3834095289 learned_addr learned my addr 172.21.15.50:0/3834095289 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:48.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4c9effd640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ca00c97d0 0x7f4ca0070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:48.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.123+0000 7f4c9e7fc640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ca015c660 0x7f4ca00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:48.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c9f7fe640 1 -- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ca015c660 msgr2=0x7f4ca00c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:48.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c9f7fe640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ca015c660 0x7f4ca00c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c9f7fe640 1 -- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ca00c97d0 msgr2=0x7f4ca0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c9f7fe640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ca00c97d0 0x7f4ca0070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c9f7fe640 1 -- 172.21.15.50:0/3834095289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880094d0 con 0x7f4ca0160240 2023-10-19T10:36:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c9f7fe640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00c26e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c680 tx=0x7f4c9000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:48.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90003b70 con 0x7f4ca0160240 2023-10-19T10:36:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90003cd0 con 0x7f4ca0160240 2023-10-19T10:36:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca00c6740 con 0x7f4ca0160240 2023-10-19T10:36:48.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90005310 con 0x7f4ca0160240 2023-10-19T10:36:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.124+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca00c6fe0 con 0x7f4ca0160240 2023-10-19T10:36:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.125+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca00c6930 con 0x7f4ca0160240 2023-10-19T10:36:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.125+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4c90005470 con 0x7f4ca0160240 2023-10-19T10:36:48.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.126+0000 7f4c97fff640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4c8006a050 0x7f4c8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:48.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.126+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4c90087cb0 con 0x7f4ca0160240 2023-10-19T10:36:48.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.126+0000 7f4c9effd640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4c8006a050 0x7f4c8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:48.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.127+0000 7f4c9effd640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4c8006a050 0x7f4c8006c510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009aa0 tx=0x7f4c98009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:48.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.131+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4c90053990 con 0x7f4ca0160240 2023-10-19T10:36:48.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:48.308+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f4ca00c6b30 con 0x7f4ca0160240 2023-10-19T10:36:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.158+0000 7f4c97fff640 1 -- 172.21.15.50:0/3834095289 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f4c90056fb0 con 0x7f4ca0160240 2023-10-19T10:36:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'rbd' created 2023-10-19T10:36:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.161+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4c8006a050 msgr2=0x7f4c8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.161+0000 7f4ca587e640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4c8006a050 0x7f4c8006c510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009aa0 tx=0x7f4c98009190 comp rx=0 tx=0).stop 2023-10-19T10:36:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 msgr2=0x7f4ca00c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00c26e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c680 tx=0x7f4c9000cb40 comp rx=0 tx=0).stop 2023-10-19T10:36:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 shutdown_connections 2023-10-19T10:36:49.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4c8006a050 0x7f4c8006c510 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ca0160240 0x7f4ca00c26e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ca015c660 0x7f4ca00c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 --2- 172.21.15.50:0/3834095289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ca00c97d0 0x7f4ca0070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 >> 172.21.15.50:0/3834095289 conn(0x7f4ca00cbc00 msgr2=0x7f4ca006dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:49.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 shutdown_connections 2023-10-19T10:36:49.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.162+0000 7f4ca587e640 1 -- 172.21.15.50:0/3834095289 wait complete. 2023-10-19T10:36:49.177 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-19T10:36:49.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.288+0000 7f9620234640 1 Processor -- start 2023-10-19T10:36:49.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.288+0000 7f9620234640 1 -- start start 2023-10-19T10:36:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f9620234640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f96180b55c0 0x7f96180cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f9620234640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f96180cf930 0x7f96180d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:49.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f9620234640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96180cfd80 con 0x7f961815c620 2023-10-19T10:36:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f9620234640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180ccc10 con 0x7f96180b55c0 2023-10-19T10:36:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f9620234640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180cda30 con 0x7f96180cf930 2023-10-19T10:36:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961d7cf640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f96180b55c0 0x7f96180cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961d7cf640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f96180b55c0 0x7f96180cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59960/0 (socket says 172.21.15.50:59960) 2023-10-19T10:36:49.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961d7cf640 1 -- 172.21.15.50:0/2696945621 learned_addr learned my addr 172.21.15.50:0/2696945621 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:49.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961dfd0640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f961815c620 0x7f96180b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48846/0 (socket says 172.21.15.50:48846) 2023-10-19T10:36:49.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961e7d1640 1 --2- 172.21.15.50:0/2696945621 >> v2:172.21.15.183:3300/0 conn(0x7f96180cf930 0x7f96180d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:49.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961d7cf640 1 -- 172.21.15.50:0/2696945621 >> v2:172.21.15.183:3300/0 conn(0x7f96180cf930 msgr2=0x7f96180d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:49.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.289+0000 7f961d7cf640 1 --2- 172.21.15.50:0/2696945621 >> v2:172.21.15.183:3300/0 conn(0x7f96180cf930 0x7f96180d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961d7cf640 1 -- 172.21.15.50:0/2696945621 >> v1:172.21.15.50:6789/0 conn(0x7f961815c620 legacy=0x7f96180b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961d7cf640 1 -- 172.21.15.50:0/2696945621 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618161110 con 0x7f96180b55c0 2023-10-19T10:36:49.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961cfce640 1 -- 172.21.15.50:0/2696945621 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226391273 0 0) 0x7f9610001110 con 0x7f961815c620 2023-10-19T10:36:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961d7cf640 1 --2- 172.21.15.50:0/2696945621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96180cf3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96180c9640 tx=0x7f960c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4788caea2d61b59d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961cfce640 1 -- 172.21.15.50:0/2696945621 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c030860 con 0x7f96180b55c0 2023-10-19T10:36:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961cfce640 1 -- 172.21.15.50:0/2696945621 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c0309c0 con 0x7f96180b55c0 2023-10-19T10:36:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.290+0000 7f961cfce640 1 -- 172.21.15.50:0/2696945621 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c030d50 con 0x7f96180b55c0 2023-10-19T10:36:49.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 -- 172.21.15.50:0/2696945621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 msgr2=0x7f96180cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:49.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 --2- 172.21.15.50:0/2696945621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96180cf3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96180c9640 tx=0x7f960c02b9a0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 -- 172.21.15.50:0/2696945621 shutdown_connections 2023-10-19T10:36:49.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 --2- 172.21.15.50:0/2696945621 >> v2:172.21.15.183:3300/0 conn(0x7f96180cf930 0x7f96180d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 --2- 172.21.15.50:0/2696945621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96180cf3f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 -- 172.21.15.50:0/2696945621 >> 172.21.15.50:0/2696945621 conn(0x7f9618071110 msgr2=0x7f96180714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:49.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 -- 172.21.15.50:0/2696945621 shutdown_connections 2023-10-19T10:36:49.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.291+0000 7f9620234640 1 -- 172.21.15.50:0/2696945621 wait complete. 2023-10-19T10:36:49.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.292+0000 7f9620234640 1 Processor -- start 2023-10-19T10:36:49.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.292+0000 7f9620234640 1 -- start start 2023-10-19T10:36:49.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f9620234640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96181f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:49.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f9620234640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96180cf930 0x7f96181543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:49.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f9620234640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961815c620 0x7f96181548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:49.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f9620234640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961815df70 con 0x7f96180b55c0 2023-10-19T10:36:49.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f9620234640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96181f2120 con 0x7f96180cf930 2023-10-19T10:36:49.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f9620234640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9618157280 con 0x7f961815c620 2023-10-19T10:36:49.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96181f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:49.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96181f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59968/0 (socket says 172.21.15.50:59968) 2023-10-19T10:36:49.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 -- 172.21.15.50:0/1057154305 learned_addr learned my addr 172.21.15.50:0/1057154305 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:49.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 -- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961815c620 msgr2=0x7f96181548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:49.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961815c620 0x7f96181548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 -- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96180cf930 msgr2=0x7f96181543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:49.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961d7cf640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96180cf930 0x7f96181543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:49.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96180cf930 0x7f96181543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:49.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961dfd0640 1 -- 172.21.15.50:0/1057154305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0094d0 con 0x7f96180b55c0 2023-10-19T10:36:49.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.293+0000 7f961d7cf640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96180cf930 0x7f96181543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:49.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.294+0000 7f961dfd0640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96181f1820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f961000c730 tx=0x7f961000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:49.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.294+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100072d0 con 0x7f96180b55c0 2023-10-19T10:36:49.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.294+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610007450 con 0x7f96180b55c0 2023-10-19T10:36:49.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.294+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9618157560 con 0x7f96180b55c0 2023-10-19T10:36:49.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.294+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100047e0 con 0x7f96180b55c0 2023-10-19T10:36:49.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.294+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961815e4a0 con 0x7f96180b55c0 2023-10-19T10:36:49.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.295+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961804e420 con 0x7f96180b55c0 2023-10-19T10:36:49.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.295+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9610004960 con 0x7f96180b55c0 2023-10-19T10:36:49.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.295+0000 7f96167fc640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f960406a050 0x7f960406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:49.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.296+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f9610087270 con 0x7f96180b55c0 2023-10-19T10:36:49.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.296+0000 7f961d7cf640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f960406a050 0x7f960406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:49.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.296+0000 7f961d7cf640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f960406a050 0x7f960406c510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f960c036000 tx=0x7f960c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:49.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.300+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9610052cd0 con 0x7f96180b55c0 2023-10-19T10:36:49.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:49.477+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f96180cfd80 con 0x7f96180b55c0 2023-10-19T10:36:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.159+0000 7f96167fc640 1 -- 172.21.15.50:0/1057154305 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f96180cfd80 con 0x7f96180b55c0 2023-10-19T10:36:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-19T10:36:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f960406a050 msgr2=0x7f960406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f960406a050 0x7f960406c510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f960c036000 tx=0x7f960c03b040 comp rx=0 tx=0).stop 2023-10-19T10:36:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 msgr2=0x7f96181f1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96181f1820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f961000c730 tx=0x7f961000cbf0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 shutdown_connections 2023-10-19T10:36:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f960406a050 0x7f960406c510 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f961815c620 0x7f96181548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96180cf930 0x7f96181543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 --2- 172.21.15.50:0/1057154305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96180b55c0 0x7f96181f1820 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 >> 172.21.15.50:0/1057154305 conn(0x7f9618071110 msgr2=0x7f96180714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:50.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 shutdown_connections 2023-10-19T10:36:50.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.163+0000 7f9620234640 1 -- 172.21.15.50:0/1057154305 wait complete. 2023-10-19T10:36:50.180 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-19T10:36:50.181 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-19T10:36:50.181 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-19T10:36:50.183 INFO:tasks.ceph.mds.a:Started 2023-10-19T10:36:50.184 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-19T10:36:50.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-19T10:36:50.186 INFO:tasks.ceph.mds.c:Started 2023-10-19T10:36:50.187 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-19T10:36:50.187 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-19T10:36:50.190 INFO:tasks.ceph.mds.b:Started 2023-10-19T10:36:50.190 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-19T10:36:50.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-19T10:36:50.193 INFO:tasks.ceph.mds.d:Started 2023-10-19T10:36:50.193 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-19T10:36:50.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:36:50.270 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2023-10-19T10:36:50.271 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2023-10-19T10:36:50.272 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-10-19T10:36:50.272 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-10-19T10:36:50.273 INFO:tasks.ceph.mds.b.smithi183.stderr:2023-10-19T10:36:50.270+0000 7f7bf0d24640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-19T10:36:50.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.308+0000 7fed2c82f640 1 Processor -- start 2023-10-19T10:36:50.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.308+0000 7fed2c82f640 1 -- start start 2023-10-19T10:36:50.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed2c82f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fed2815c570 0x7fed280d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed2c82f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed280cd920 0x7fed280d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed2c82f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2806aa30 con 0x7fed2815c950 2023-10-19T10:36:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed2c82f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2806b2d0 con 0x7fed2815c570 2023-10-19T10:36:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed2c82f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280ce090 con 0x7fed280cd920 2023-10-19T10:36:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed2659c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fed2815c570 0x7fed280d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed26d9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed280cd920 0x7fed280d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed25d9b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fed2815c950 0x7fed2806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:47264/0 (socket says 172.21.15.50:47264) 2023-10-19T10:36:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.309+0000 7fed25d9b640 1 -- 172.21.15.50:0/945002586 learned_addr learned my addr 172.21.15.50:0/945002586 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed2559a640 1 -- 172.21.15.50:0/945002586 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887198059 0 0) 0x7fed2806aa30 con 0x7fed2815c950 2023-10-19T10:36:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed2559a640 1 -- 172.21.15.50:0/945002586 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed10003520 con 0x7fed2815c950 2023-10-19T10:36:50.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed26d9d640 1 -- 172.21.15.50:0/945002586 >> v2:172.21.15.183:3300/0 conn(0x7fed2815c570 msgr2=0x7fed280d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:50.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed26d9d640 1 --2- 172.21.15.50:0/945002586 >> v2:172.21.15.183:3300/0 conn(0x7fed2815c570 0x7fed280d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed26d9d640 1 -- 172.21.15.50:0/945002586 >> v1:172.21.15.183:6789/0 conn(0x7fed2815c950 legacy=0x7fed2806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed26d9d640 1 -- 172.21.15.50:0/945002586 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2806ac80 con 0x7fed280cd920 2023-10-19T10:36:50.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed2659c640 1 --2- 172.21.15.50:0/945002586 >> v2:172.21.15.183:3300/0 conn(0x7fed2815c570 0x7fed280d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:50.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.310+0000 7fed26d9d640 1 --2- 172.21.15.50:0/945002586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed280cd920 0x7fed280d26a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fed20009d90 tx=0x7fed2002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e0ece6f8416510 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2559a640 1 -- 172.21.15.50:0/945002586 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2002cbc0 con 0x7fed280cd920 2023-10-19T10:36:50.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2559a640 1 -- 172.21.15.50:0/945002586 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2002cd20 con 0x7fed280cd920 2023-10-19T10:36:50.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2559a640 1 -- 172.21.15.50:0/945002586 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20034640 con 0x7fed280cd920 2023-10-19T10:36:50.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 -- 172.21.15.50:0/945002586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed280cd920 msgr2=0x7fed280d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 --2- 172.21.15.50:0/945002586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed280cd920 0x7fed280d26a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fed20009d90 tx=0x7fed2002be10 comp rx=0 tx=0).stop 2023-10-19T10:36:50.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 -- 172.21.15.50:0/945002586 shutdown_connections 2023-10-19T10:36:50.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 --2- 172.21.15.50:0/945002586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed280cd920 0x7fed280d26a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 --2- 172.21.15.50:0/945002586 >> v2:172.21.15.183:3300/0 conn(0x7fed2815c570 0x7fed280d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 -- 172.21.15.50:0/945002586 >> 172.21.15.50:0/945002586 conn(0x7fed28089ff0 msgr2=0x7fed2809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:50.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 -- 172.21.15.50:0/945002586 shutdown_connections 2023-10-19T10:36:50.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.311+0000 7fed2c82f640 1 -- 172.21.15.50:0/945002586 wait complete. 2023-10-19T10:36:50.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.312+0000 7fed2c82f640 1 Processor -- start 2023-10-19T10:36:50.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 -- start start 2023-10-19T10:36:50.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 0x7fed2814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed2815c570 0x7fed2815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed2815c950 0x7fed2815f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2815d350 con 0x7fed280cd920 2023-10-19T10:36:50.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2815a190 con 0x7fed2815c950 2023-10-19T10:36:50.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2c82f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2814e180 con 0x7fed2815c570 2023-10-19T10:36:50.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 0x7fed2814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 0x7fed2814dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60032/0 (socket says 172.21.15.50:60032) 2023-10-19T10:36:50.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed2659c640 1 -- 172.21.15.50:0/2583108507 learned_addr learned my addr 172.21.15.50:0/2583108507 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:50.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed25d9b640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed2815c570 0x7fed2815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.313+0000 7fed26d9d640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed2815c950 0x7fed2815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2659c640 1 -- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed2815c570 msgr2=0x7fed2815f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2659c640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed2815c570 0x7fed2815f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2659c640 1 -- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed2815c950 msgr2=0x7fed2815f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2659c640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed2815c950 0x7fed2815f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2659c640 1 -- 172.21.15.50:0/2583108507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180093a0 con 0x7fed280cd920 2023-10-19T10:36:50.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2659c640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 0x7fed2814dc40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed1800ebe0 tx=0x7fed1800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18009750 con 0x7fed280cd920 2023-10-19T10:36:50.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1800cc30 con 0x7fed280cd920 2023-10-19T10:36:50.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed20009a70 con 0x7fed280cd920 2023-10-19T10:36:50.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800eee0 con 0x7fed280cd920 2023-10-19T10:36:50.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.314+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed280ca6a0 con 0x7fed280cd920 2023-10-19T10:36:50.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.315+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed28073990 con 0x7fed280cd920 2023-10-19T10:36:50.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.316+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fed18005990 con 0x7fed280cd920 2023-10-19T10:36:50.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.316+0000 7fed16ffd640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fed0806a050 0x7fed0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.316+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fed180886e0 con 0x7fed280cd920 2023-10-19T10:36:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.316+0000 7fed25d9b640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fed0806a050 0x7fed0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.317+0000 7fed25d9b640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fed0806a050 0x7fed0806c510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed2815a690 tx=0x7fed1c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.320+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fed18054260 con 0x7fed280cd920 2023-10-19T10:36:50.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.498+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed2815d5c0 con 0x7fed280cd920 2023-10-19T10:36:50.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.499+0000 7fed16ffd640 1 -- 172.21.15.50:0/2583108507 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14613 (secure 0 0 0) 0x7fed18057880 con 0x7fed280cd920 2023-10-19T10:36:50.502 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:36:50.502 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":18,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:50.157081+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:36:50.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fed0806a050 msgr2=0x7fed0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fed0806a050 0x7fed0806c510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed2815a690 tx=0x7fed1c007d30 comp rx=0 tx=0).stop 2023-10-19T10:36:50.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 msgr2=0x7fed2814dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 0x7fed2814dc40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed1800ebe0 tx=0x7fed1800c520 comp rx=0 tx=0).stop 2023-10-19T10:36:50.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 shutdown_connections 2023-10-19T10:36:50.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fed0806a050 0x7fed0806c510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed2815c950 0x7fed2815f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed2815c570 0x7fed2815f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 --2- 172.21.15.50:0/2583108507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed280cd920 0x7fed2814dc40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 >> 172.21.15.50:0/2583108507 conn(0x7fed28089ff0 msgr2=0x7fed2815bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:50.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 shutdown_connections 2023-10-19T10:36:50.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.501+0000 7fed2c82f640 1 -- 172.21.15.50:0/2583108507 wait complete. 2023-10-19T10:36:50.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:36:50.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:36:50.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.621+0000 7f3d5fb85640 1 Processor -- start 2023-10-19T10:36:50.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.621+0000 7f3d5fb85640 1 -- start start 2023-10-19T10:36:50.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.622+0000 7f3d5fb85640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3d580cf1f0 0x7f3d580cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.622+0000 7f3d5fb85640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580cfb20 con 0x7f3d580cada0 2023-10-19T10:36:50.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.622+0000 7f3d5fb85640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580cfe00 con 0x7f3d580b1d10 2023-10-19T10:36:50.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.622+0000 7f3d5fb85640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5815a8b0 con 0x7f3d580cf1f0 2023-10-19T10:36:50.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.622+0000 7f3d5d120640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3d580cada0 0x7f3d5815b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48878/0 (socket says 172.21.15.50:48878) 2023-10-19T10:36:50.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.622+0000 7f3d5d120640 1 -- 172.21.15.50:0/981984881 learned_addr learned my addr 172.21.15.50:0/981984881 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:50.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5e122640 1 --2- 172.21.15.50:0/981984881 >> v2:172.21.15.183:3300/0 conn(0x7f3d580cf1f0 0x7f3d580cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901784219 0 0) 0x7f3d580cfb20 con 0x7f3d580cada0 2023-10-19T10:36:50.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d40003520 con 0x7f3d580cada0 2023-10-19T10:36:50.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105616107 0 0) 0x7f3d40003520 con 0x7f3d580cada0 2023-10-19T10:36:50.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 >> v2:172.21.15.183:3300/0 conn(0x7f3d580cf1f0 msgr2=0x7f3d580cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:50.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 --2- 172.21.15.50:0/981984881 >> v2:172.21.15.183:3300/0 conn(0x7f3d580cf1f0 0x7f3d580cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 >> v1:172.21.15.183:6789/0 conn(0x7f3d580b1d10 legacy=0x7f3d580b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5815aac0 con 0x7f3d580cada0 2023-10-19T10:36:50.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f3d4c001670 con 0x7f3d580cada0 2023-10-19T10:36:50.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 >> v1:172.21.15.50:6789/0 conn(0x7f3d580cada0 legacy=0x7f3d5815b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40003970 0x7f3d40003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d40004370 0x7f3d40004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d400050e0 0x7f3d4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5815a8b0 con 0x7f3d40004370 2023-10-19T10:36:50.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4000c020 con 0x7f3d40003970 2023-10-19T10:36:50.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4000c1e0 con 0x7f3d400050e0 2023-10-19T10:36:50.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d120640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d40004370 0x7f3d40004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d921640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40003970 0x7f3d40003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5e122640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d400050e0 0x7f3d4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d120640 1 -- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d400050e0 msgr2=0x7f3d4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:50.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d120640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d400050e0 0x7f3d4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d120640 1 -- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40003970 msgr2=0x7f3d40003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:50.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d120640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40003970 0x7f3d40003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5d120640 1 -- 172.21.15.50:0/981984881 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d40003520 con 0x7f3d40004370 2023-10-19T10:36:50.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.623+0000 7f3d5e122640 1 -- 172.21.15.50:0/981984881 reap_dead start 2023-10-19T10:36:50.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.624+0000 7f3d5d120640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d40004370 0x7f3d40004840 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c190 tx=0x7f3d400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d42f431b8535e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.624+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c005ad0 con 0x7f3d40004370 2023-10-19T10:36:50.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.624+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c005c50 con 0x7f3d40004370 2023-10-19T10:36:50.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.624+0000 7f3d5c91f640 1 -- 172.21.15.50:0/981984881 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c02c7c0 con 0x7f3d40004370 2023-10-19T10:36:50.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 -- 172.21.15.50:0/981984881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d40004370 msgr2=0x7f3d40004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d40004370 0x7f3d40004840 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c190 tx=0x7f3d400057f0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 -- 172.21.15.50:0/981984881 shutdown_connections 2023-10-19T10:36:50.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/981984881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d40004370 0x7f3d40004840 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 -- 172.21.15.50:0/981984881 >> 172.21.15.50:0/981984881 conn(0x7f3d580ccd10 msgr2=0x7f3d580cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:50.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 -- 172.21.15.50:0/981984881 shutdown_connections 2023-10-19T10:36:50.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.625+0000 7f3d5fb85640 1 -- 172.21.15.50:0/981984881 wait complete. 2023-10-19T10:36:50.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.626+0000 7f3d5fb85640 1 Processor -- start 2023-10-19T10:36:50.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.626+0000 7f3d5fb85640 1 -- start start 2023-10-19T10:36:50.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5fb85640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40004370 0x7f3d58156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5fb85640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d581488d0 0x7f3d58157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5fb85640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 0x7f3d5806b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5fb85640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58157a10 con 0x7f3d581517e0 2023-10-19T10:36:50.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5fb85640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58156d90 con 0x7f3d40004370 2023-10-19T10:36:50.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5fb85640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58157070 con 0x7f3d581488d0 2023-10-19T10:36:50.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 0x7f3d5806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 0x7f3d5806b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60050/0 (socket says 172.21.15.50:60050) 2023-10-19T10:36:50.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 -- 172.21.15.50:0/3528425250 learned_addr learned my addr 172.21.15.50:0/3528425250 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:50.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5d921640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40004370 0x7f3d58156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5d120640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d581488d0 0x7f3d58157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 -- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d581488d0 msgr2=0x7f3d58157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d581488d0 0x7f3d58157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 -- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40004370 msgr2=0x7f3d58156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40004370 0x7f3d58156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 -- 172.21.15.50:0/3528425250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d58157350 con 0x7f3d581517e0 2023-10-19T10:36:50.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d5e122640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 0x7f3d5806b960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d580cddf0 tx=0x7f3d4800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.627+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800e040 con 0x7f3d581517e0 2023-10-19T10:36:50.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.628+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4800f040 con 0x7f3d581517e0 2023-10-19T10:36:50.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.628+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d58157630 con 0x7f3d581517e0 2023-10-19T10:36:50.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.628+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48015430 con 0x7f3d581517e0 2023-10-19T10:36:50.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.628+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5806bea0 con 0x7f3d581517e0 2023-10-19T10:36:50.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.629+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d58158370 con 0x7f3d581517e0 2023-10-19T10:36:50.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.629+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3d48015590 con 0x7f3d581517e0 2023-10-19T10:36:50.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.629+0000 7f3d567fc640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d4406a050 0x7f3d4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.629+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3d480888e0 con 0x7f3d581517e0 2023-10-19T10:36:50.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.629+0000 7f3d5d921640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d4406a050 0x7f3d4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.630+0000 7f3d5d921640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d4406a050 0x7f3d4406c510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d58068660 tx=0x7f3d50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.634+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3d480543d0 con 0x7f3d581517e0 2023-10-19T10:36:50.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.808+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d58157b50 con 0x7f3d581517e0 2023-10-19T10:36:50.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.809+0000 7f3d567fc640 1 -- 172.21.15.50:0/3528425250 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d58157b50 con 0x7f3d581517e0 2023-10-19T10:36:50.812 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:36:50.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.810+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d4406a050 msgr2=0x7f3d4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d4406a050 0x7f3d4406c510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d58068660 tx=0x7f3d50009040 comp rx=0 tx=0).stop 2023-10-19T10:36:50.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 msgr2=0x7f3d5806b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 0x7f3d5806b960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d580cddf0 tx=0x7f3d4800f580 comp rx=0 tx=0).stop 2023-10-19T10:36:50.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 shutdown_connections 2023-10-19T10:36:50.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d4406a050 0x7f3d4406c510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d581517e0 0x7f3d5806b960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d581488d0 0x7f3d58157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 --2- 172.21.15.50:0/3528425250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d40004370 0x7f3d58156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 >> 172.21.15.50:0/3528425250 conn(0x7f3d580ccd10 msgr2=0x7f3d580d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 shutdown_connections 2023-10-19T10:36:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.811+0000 7f3d5fb85640 1 -- 172.21.15.50:0/3528425250 wait complete. 2023-10-19T10:36:50.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:36:50.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.929+0000 7fe487058640 1 Processor -- start 2023-10-19T10:36:50.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.929+0000 7fe487058640 1 -- start start 2023-10-19T10:36:50.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe487058640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe4800d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe487058640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe4800b3090 0x7fe480160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe487058640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4800af250 con 0x7fe4800b3cd0 2023-10-19T10:36:50.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe487058640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4800b0190 con 0x7fe4800b3090 2023-10-19T10:36:50.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe487058640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800af510 con 0x7fe4800b38f0 2023-10-19T10:36:50.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe4800b3090 0x7fe480160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe4800b3090 0x7fe480160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60056/0 (socket says 172.21.15.50:60056) 2023-10-19T10:36:50.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe4855f5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe4800b3cd0 0x7fe4800b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48892/0 (socket says 172.21.15.50:48892) 2023-10-19T10:36:50.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 -- 172.21.15.50:0/3570296849 learned_addr learned my addr 172.21.15.50:0/3570296849 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:50.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe484df4640 1 --2- 172.21.15.50:0/3570296849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe4800d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 -- 172.21.15.50:0/3570296849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 msgr2=0x7fe4800d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:50.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 --2- 172.21.15.50:0/3570296849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe4800d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 -- 172.21.15.50:0/3570296849 >> v1:172.21.15.50:6789/0 conn(0x7fe4800b3cd0 legacy=0x7fe4800b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47ffff640 1 -- 172.21.15.50:0/3570296849 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800af7f0 con 0x7fe4800b3090 2023-10-19T10:36:50.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.930+0000 7fe47f7fe640 1 -- 172.21.15.50:0/3570296849 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643433251 0 0) 0x7fe4800af250 con 0x7fe4800b3cd0 2023-10-19T10:36:50.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.931+0000 7fe47ffff640 1 --2- 172.21.15.50:0/3570296849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3090 0x7fe480160b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009870 tx=0x7fe46c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15405b2e13753e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.931+0000 7fe47f7fe640 1 -- 172.21.15.50:0/3570296849 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c030840 con 0x7fe4800b3090 2023-10-19T10:36:50.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.931+0000 7fe47f7fe640 1 -- 172.21.15.50:0/3570296849 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c0309a0 con 0x7fe4800b3090 2023-10-19T10:36:50.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.931+0000 7fe47f7fe640 1 -- 172.21.15.50:0/3570296849 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c030d30 con 0x7fe4800b3090 2023-10-19T10:36:50.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 -- 172.21.15.50:0/3570296849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3090 msgr2=0x7fe480160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 --2- 172.21.15.50:0/3570296849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3090 0x7fe480160b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009870 tx=0x7fe46c02b980 comp rx=0 tx=0).stop 2023-10-19T10:36:50.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 -- 172.21.15.50:0/3570296849 shutdown_connections 2023-10-19T10:36:50.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 --2- 172.21.15.50:0/3570296849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3090 0x7fe480160b00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 --2- 172.21.15.50:0/3570296849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe4800d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 -- 172.21.15.50:0/3570296849 >> 172.21.15.50:0/3570296849 conn(0x7fe48009b960 msgr2=0x7fe4800d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:50.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 -- 172.21.15.50:0/3570296849 shutdown_connections 2023-10-19T10:36:50.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.932+0000 7fe487058640 1 -- 172.21.15.50:0/3570296849 wait complete. 2023-10-19T10:36:50.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.933+0000 7fe487058640 1 Processor -- start 2023-10-19T10:36:50.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.933+0000 7fe487058640 1 -- start start 2023-10-19T10:36:50.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4800b3090 0x7fe4800b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe48014dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 0x7fe48006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800b51e0 con 0x7fe4800b3cd0 2023-10-19T10:36:50.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800b45d0 con 0x7fe4800b3090 2023-10-19T10:36:50.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800b48b0 con 0x7fe4800b38f0 2023-10-19T10:36:50.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 0x7fe48006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 0x7fe48006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60072/0 (socket says 172.21.15.50:60072) 2023-10-19T10:36:50.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 -- 172.21.15.50:0/4028065830 learned_addr learned my addr 172.21.15.50:0/4028065830 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:50.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe484df4640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4800b3090 0x7fe4800b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe47ffff640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe48014dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 -- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 msgr2=0x7fe48014dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe48014dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 -- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4800b3090 msgr2=0x7fe4800b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:50.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4800b3090 0x7fe4800b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:50.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 -- 172.21.15.50:0/4028065830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0094d0 con 0x7fe4800b3cd0 2023-10-19T10:36:50.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe4855f5640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 0x7fe48006b120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe47800c780 tx=0x7fe47800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800ce90 con 0x7fe4800b3cd0 2023-10-19T10:36:50.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47800a640 con 0x7fe4800b3cd0 2023-10-19T10:36:50.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800a9d0 con 0x7fe4800b3cd0 2023-10-19T10:36:50.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.934+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4800b4b90 con 0x7fe4800b3cd0 2023-10-19T10:36:50.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.935+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4800b7f90 con 0x7fe4800b3cd0 2023-10-19T10:36:50.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.935+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4800b5350 con 0x7fe4800b3cd0 2023-10-19T10:36:50.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.936+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe47800ab30 con 0x7fe4800b3cd0 2023-10-19T10:36:50.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.936+0000 7fe47d7fa640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe46406a050 0x7fe46406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:50.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.936+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe4780876e0 con 0x7fe4800b3cd0 2023-10-19T10:36:50.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.936+0000 7fe484df4640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe46406a050 0x7fe46406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:50.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.937+0000 7fe484df4640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe46406a050 0x7fe46406c510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe474009b30 tx=0x7fe474009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:50.941+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe4780531d0 con 0x7fe4800b3cd0 2023-10-19T10:36:51.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.117+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe48004e420 con 0x7fe4800b3cd0 2023-10-19T10:36:51.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.118+0000 7fe47d7fa640 1 -- 172.21.15.50:0/4028065830 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe4780180a0 con 0x7fe4800b3cd0 2023-10-19T10:36:51.121 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:36:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe46406a050 msgr2=0x7fe46406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe46406a050 0x7fe46406c510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe474009b30 tx=0x7fe474009110 comp rx=0 tx=0).stop 2023-10-19T10:36:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 msgr2=0x7fe48006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 0x7fe48006b120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe47800c780 tx=0x7fe47800cc40 comp rx=0 tx=0).stop 2023-10-19T10:36:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 shutdown_connections 2023-10-19T10:36:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe46406a050 0x7fe46406c510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4800b3cd0 0x7fe48006b120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4800b38f0 0x7fe48014dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 --2- 172.21.15.50:0/4028065830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4800b3090 0x7fe4800b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.119+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 >> 172.21.15.50:0/4028065830 conn(0x7fe48009b960 msgr2=0x7fe48006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.120+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 shutdown_connections 2023-10-19T10:36:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.120+0000 7fe487058640 1 -- 172.21.15.50:0/4028065830 wait complete. 2023-10-19T10:36:51.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:36:51.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.237+0000 7f1d7ca92640 1 Processor -- start 2023-10-19T10:36:51.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.237+0000 7f1d7ca92640 1 -- start start 2023-10-19T10:36:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7ca92640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1d78069f10 0x7f1d7815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7ca92640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1d780c27d0 0x7f1d78070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7ca92640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7815f050 con 0x7f1d7815b780 2023-10-19T10:36:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7ca92640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d780cb990 con 0x7f1d780c27d0 2023-10-19T10:36:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7ca92640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7815e0c0 con 0x7f1d78069f10 2023-10-19T10:36:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7659c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1d7815b780 0x7f1d78069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48906/0 (socket says 172.21.15.50:48906) 2023-10-19T10:36:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d7659c640 1 -- 172.21.15.50:0/2175617338 learned_addr learned my addr 172.21.15.50:0/2175617338 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.238+0000 7f1d76d9d640 1 --2- 172.21.15.50:0/2175617338 >> v2:172.21.15.50:3300/0 conn(0x7f1d780c27d0 0x7f1d78070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d75d9b640 1 --2- 172.21.15.50:0/2175617338 >> v2:172.21.15.183:3300/0 conn(0x7f1d78069f10 0x7f1d7815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d76d9d640 1 -- 172.21.15.50:0/2175617338 >> v2:172.21.15.183:3300/0 conn(0x7f1d78069f10 msgr2=0x7f1d7815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d76d9d640 1 --2- 172.21.15.50:0/2175617338 >> v2:172.21.15.183:3300/0 conn(0x7f1d78069f10 0x7f1d7815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d76d9d640 1 -- 172.21.15.50:0/2175617338 >> v1:172.21.15.50:6789/0 conn(0x7f1d7815b780 legacy=0x7f1d78069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d7559a640 1 -- 172.21.15.50:0/2175617338 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758161467 0 0) 0x7f1d68001490 con 0x7f1d7815b780 2023-10-19T10:36:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d76d9d640 1 -- 172.21.15.50:0/2175617338 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d780cc850 con 0x7f1d780c27d0 2023-10-19T10:36:51.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.239+0000 7f1d76d9d640 1 --2- 172.21.15.50:0/2175617338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d78070780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d7806d910 tx=0x7f1d7002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4612c665dcf49850 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7559a640 1 -- 172.21.15.50:0/2175617338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70031710 con 0x7f1d780c27d0 2023-10-19T10:36:51.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7559a640 1 -- 172.21.15.50:0/2175617338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7002f050 con 0x7f1d780c27d0 2023-10-19T10:36:51.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7559a640 1 -- 172.21.15.50:0/2175617338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70031a90 con 0x7f1d780c27d0 2023-10-19T10:36:51.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 -- 172.21.15.50:0/2175617338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 msgr2=0x7f1d78070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/2175617338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d78070780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d7806d910 tx=0x7f1d7002b7b0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 -- 172.21.15.50:0/2175617338 shutdown_connections 2023-10-19T10:36:51.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/2175617338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d78070780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/2175617338 >> v2:172.21.15.183:3300/0 conn(0x7f1d78069f10 0x7f1d7815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 -- 172.21.15.50:0/2175617338 >> 172.21.15.50:0/2175617338 conn(0x7f1d780cbbc0 msgr2=0x7f1d780cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:51.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 -- 172.21.15.50:0/2175617338 shutdown_connections 2023-10-19T10:36:51.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.240+0000 7f1d7ca92640 1 -- 172.21.15.50:0/2175617338 wait complete. 2023-10-19T10:36:51.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.241+0000 7f1d7ca92640 1 Processor -- start 2023-10-19T10:36:51.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 -- start start 2023-10-19T10:36:51.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d78069f10 0x7f1d780c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d780ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d7815b780 0x7f1d780ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780cd6d0 con 0x7f1d780c27d0 2023-10-19T10:36:51.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780cd9b0 con 0x7f1d78069f10 2023-10-19T10:36:51.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7ca92640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780cdc90 con 0x7f1d7815b780 2023-10-19T10:36:51.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d75d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d780ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d75d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d780ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60096/0 (socket says 172.21.15.50:60096) 2023-10-19T10:36:51.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d75d9b640 1 -- 172.21.15.50:0/3231115147 learned_addr learned my addr 172.21.15.50:0/3231115147 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:51.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.242+0000 7f1d7659c640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d78069f10 0x7f1d780c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d76d9d640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d7815b780 0x7f1d780ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d75d9b640 1 -- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d7815b780 msgr2=0x7f1d780ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d75d9b640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d7815b780 0x7f1d780ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d75d9b640 1 -- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d78069f10 msgr2=0x7f1d780c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d75d9b640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d78069f10 0x7f1d780c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d75d9b640 1 -- 172.21.15.50:0/3231115147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d70009400 con 0x7f1d780c27d0 2023-10-19T10:36:51.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d7659c640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d78069f10 0x7f1d780c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:51.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d76d9d640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d7815b780 0x7f1d780ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:51.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d75d9b640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d780ce770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000d7d0 tx=0x7f1d6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000b6c0 con 0x7f1d780c27d0 2023-10-19T10:36:51.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6000b820 con 0x7f1d780c27d0 2023-10-19T10:36:51.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d780cdf70 con 0x7f1d780c27d0 2023-10-19T10:36:51.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000bbb0 con 0x7f1d780c27d0 2023-10-19T10:36:51.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.243+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d780cecb0 con 0x7f1d780c27d0 2023-10-19T10:36:51.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.244+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d78071c70 con 0x7f1d780c27d0 2023-10-19T10:36:51.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.244+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d6000bd10 con 0x7f1d780c27d0 2023-10-19T10:36:51.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.245+0000 7f1d6effd640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1d5806a050 0x7f1d5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.245+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1d60088770 con 0x7f1d780c27d0 2023-10-19T10:36:51.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.245+0000 7f1d7659c640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1d5806a050 0x7f1d5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.245+0000 7f1d7659c640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1d5806a050 0x7f1d5806c510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d680012e0 tx=0x7f1d68005b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.249+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1d60057c40 con 0x7f1d780c27d0 2023-10-19T10:36:51.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.462+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d78073990 con 0x7f1d780c27d0 2023-10-19T10:36:51.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.462+0000 7f1d6effd640 1 -- 172.21.15.50:0/3231115147 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f1d600577d0 con 0x7f1d780c27d0 2023-10-19T10:36:51.466 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:36:51.466 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-19T10:36:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 3 2023-10-19T10:36:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1d5806a050 msgr2=0x7f1d5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1d5806a050 0x7f1d5806c510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d680012e0 tx=0x7f1d68005b90 comp rx=0 tx=0).stop 2023-10-19T10:36:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 msgr2=0x7f1d780ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d780ce770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000d7d0 tx=0x7f1d6000dc90 comp rx=0 tx=0).stop 2023-10-19T10:36:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 shutdown_connections 2023-10-19T10:36:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1d5806a050 0x7f1d5806c510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d7815b780 0x7f1d780ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d780c27d0 0x7f1d780ce770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 --2- 172.21.15.50:0/3231115147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d78069f10 0x7f1d780c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 >> 172.21.15.50:0/3231115147 conn(0x7f1d780cbbc0 msgr2=0x7f1d78070680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 shutdown_connections 2023-10-19T10:36:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.464+0000 7f1d7ca92640 1 -- 172.21.15.50:0/3231115147 wait complete. 2023-10-19T10:36:51.477 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:36:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.580+0000 7fa7ba9f7640 1 Processor -- start 2023-10-19T10:36:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.580+0000 7fa7ba9f7640 1 -- start start 2023-10-19T10:36:51.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7ba9f7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa7b415c6e0 0x7fa7b40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7ba9f7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa7b40d1cd0 0x7fa7b40cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7ba9f7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b4160670 con 0x7fa7b40b1860 2023-10-19T10:36:51.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7ba9f7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b4161280 con 0x7fa7b40d1cd0 2023-10-19T10:36:51.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7ba9f7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b4160950 con 0x7fa7b415c6e0 2023-10-19T10:36:51.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7b8f94640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa7b40d1cd0 0x7fa7b40cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7b8f94640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa7b40d1cd0 0x7fa7b40cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60102/0 (socket says 172.21.15.50:60102) 2023-10-19T10:36:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2861118145 learned_addr learned my addr 172.21.15.50:0/2861118145 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.581+0000 7fa7b37fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa7b40b1860 0x7fa7b40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48910/0 (socket says 172.21.15.50:48910) 2023-10-19T10:36:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b3fff640 1 --2- 172.21.15.50:0/2861118145 >> v2:172.21.15.183:3300/0 conn(0x7fa7b415c6e0 0x7fa7b40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2861118145 >> v2:172.21.15.183:3300/0 conn(0x7fa7b415c6e0 msgr2=0x7fa7b40cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b8f94640 1 --2- 172.21.15.50:0/2861118145 >> v2:172.21.15.183:3300/0 conn(0x7fa7b415c6e0 0x7fa7b40cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2861118145 >> v1:172.21.15.50:6789/0 conn(0x7fa7b40b1860 legacy=0x7fa7b40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2861118145 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4160c30 con 0x7fa7b40d1cd0 2023-10-19T10:36:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b2ffd640 1 -- 172.21.15.50:0/2861118145 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884544667 0 0) 0x7fa7b4160670 con 0x7fa7b40b1860 2023-10-19T10:36:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b3fff640 1 --2- 172.21.15.50:0/2861118145 >> v2:172.21.15.183:3300/0 conn(0x7fa7b415c6e0 0x7fa7b40cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.582+0000 7fa7b8f94640 1 --2- 172.21.15.50:0/2861118145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40cd850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40cfaa0 tx=0x7fa7ac02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5035b846ad0bd33 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7b2ffd640 1 -- 172.21.15.50:0/2861118145 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac02c9f0 con 0x7fa7b40d1cd0 2023-10-19T10:36:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7b2ffd640 1 -- 172.21.15.50:0/2861118145 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac02cb50 con 0x7fa7b40d1cd0 2023-10-19T10:36:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7b2ffd640 1 -- 172.21.15.50:0/2861118145 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0343f0 con 0x7fa7b40d1cd0 2023-10-19T10:36:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2861118145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b40d1cd0 msgr2=0x7fa7b40cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2861118145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40cd850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40cfaa0 tx=0x7fa7ac02be50 comp rx=0 tx=0).stop 2023-10-19T10:36:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2861118145 shutdown_connections 2023-10-19T10:36:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2861118145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40cd850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2861118145 >> v2:172.21.15.183:3300/0 conn(0x7fa7b415c6e0 0x7fa7b40cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2861118145 >> 172.21.15.50:0/2861118145 conn(0x7fa7b406a530 msgr2=0x7fa7b406a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2861118145 shutdown_connections 2023-10-19T10:36:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.583+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2861118145 wait complete. 2023-10-19T10:36:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.584+0000 7fa7ba9f7640 1 Processor -- start 2023-10-19T10:36:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.584+0000 7fa7ba9f7640 1 -- start start 2023-10-19T10:36:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7ba9f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b40b1860 0x7fa7b4150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7ba9f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7ba9f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 0x7fa7b40c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7ba9f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4153c40 con 0x7fa7b415c6e0 2023-10-19T10:36:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7ba9f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41537b0 con 0x7fa7b40d1cd0 2023-10-19T10:36:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7ba9f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40b3cd0 con 0x7fa7b40b1860 2023-10-19T10:36:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7b8f94640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 0x7fa7b40c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7b8f94640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 0x7fa7b40c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60114/0 (socket says 172.21.15.50:60114) 2023-10-19T10:36:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2651557274 learned_addr learned my addr 172.21.15.50:0/2651557274 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7b3fff640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b40b1860 0x7fa7b4150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.585+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b40b1860 msgr2=0x7fa7b4150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b37fe640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b8f94640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b40b1860 0x7fa7b4150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b40d1cd0 msgr2=0x7fa7b40b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b8f94640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b8f94640 1 -- 172.21.15.50:0/2651557274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac009b30 con 0x7fa7b415c6e0 2023-10-19T10:36:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b3fff640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b40b1860 0x7fa7b4150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b37fe640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b8f94640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 0x7fa7b40c7f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac036040 tx=0x7fa7ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac009510 con 0x7fa7b415c6e0 2023-10-19T10:36:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac03e450 con 0x7fa7b415c6e0 2023-10-19T10:36:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac03e7e0 con 0x7fa7b415c6e0 2023-10-19T10:36:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b41f1d40 con 0x7fa7b415c6e0 2023-10-19T10:36:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.586+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b4152e80 con 0x7fa7b415c6e0 2023-10-19T10:36:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.587+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b4153940 con 0x7fa7b415c6e0 2023-10-19T10:36:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.587+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa7ac03e940 con 0x7fa7b415c6e0 2023-10-19T10:36:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.588+0000 7fa7b0ff9640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa7a006a050 0x7fa7a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.588+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7ac0ab080 con 0x7fa7b415c6e0 2023-10-19T10:36:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.588+0000 7fa7b3fff640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa7a006a050 0x7fa7a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.588+0000 7fa7b3fff640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa7a006a050 0x7fa7a006c510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006fd0 tx=0x7fa7a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.592+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa7ac0dd940 con 0x7fa7b415c6e0 2023-10-19T10:36:51.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.768+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7b4073990 con 0x7fa7b415c6e0 2023-10-19T10:36:51.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.769+0000 7fa7b0ff9640 1 -- 172.21.15.50:0/2651557274 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14600 (secure 0 0 0) 0x7fa7ac043020 con 0x7fa7b415c6e0 2023-10-19T10:36:51.772 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:36:51.773 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":19,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:51.158414+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:36:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa7a006a050 msgr2=0x7fa7a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa7a006a050 0x7fa7a006c510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006fd0 tx=0x7fa7a8008040 comp rx=0 tx=0).stop 2023-10-19T10:36:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 msgr2=0x7fa7b40c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 0x7fa7b40c7f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac036040 tx=0x7fa7ac004a40 comp rx=0 tx=0).stop 2023-10-19T10:36:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 shutdown_connections 2023-10-19T10:36:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa7a006a050 0x7fa7a006c510 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7b415c6e0 0x7fa7b40c7f10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b40d1cd0 0x7fa7b40b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 --2- 172.21.15.50:0/2651557274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b40b1860 0x7fa7b4150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 >> 172.21.15.50:0/2651557274 conn(0x7fa7b406a530 msgr2=0x7fa7b415a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 shutdown_connections 2023-10-19T10:36:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.771+0000 7fa7ba9f7640 1 -- 172.21.15.50:0/2651557274 wait complete. 2023-10-19T10:36:51.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:36:51.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:36:51.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.891+0000 7efcfdae3640 1 Processor -- start 2023-10-19T10:36:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.891+0000 7efcfdae3640 1 -- start start 2023-10-19T10:36:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcfdae3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 0x7efcf80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcfdae3640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efcf8161210 0x7efcf80b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcfdae3640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80af250 con 0x7efcf80b3090 2023-10-19T10:36:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcfdae3640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80b0190 con 0x7efcf8161210 2023-10-19T10:36:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcfdae3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80af510 con 0x7efcf80b38f0 2023-10-19T10:36:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcf6ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7efcf80b3090 0x7efcf8160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48918/0 (socket says 172.21.15.50:48918) 2023-10-19T10:36:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcf6ffd640 1 -- 172.21.15.50:0/1333788206 learned_addr learned my addr 172.21.15.50:0/1333788206 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcf77fe640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 0x7efcf80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcf7fff640 1 --2- 172.21.15.50:0/1333788206 >> v2:172.21.15.183:3300/0 conn(0x7efcf8161210 0x7efcf80b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463373052 0 0) 0x7efcf80af250 con 0x7efcf80b3090 2023-10-19T10:36:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.892+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce0003520 con 0x7efcf80b3090 2023-10-19T10:36:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1838125934 0 0) 0x7efce0003520 con 0x7efcf80b3090 2023-10-19T10:36:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 msgr2=0x7efcf80d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 0x7efcf80d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 >> v2:172.21.15.183:3300/0 conn(0x7efcf8161210 msgr2=0x7efcf80b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 --2- 172.21.15.50:0/1333788206 >> v2:172.21.15.183:3300/0 conn(0x7efcf8161210 0x7efcf80b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80af7f0 con 0x7efcf80b3090 2023-10-19T10:36:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7efcec001670 con 0x7efcf80b3090 2023-10-19T10:36:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 >> v1:172.21.15.50:6789/0 conn(0x7efcf80b3090 legacy=0x7efcf8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 0x7efce0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce0003ef0 0x7efce00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0004d00 0x7efce0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80b0190 con 0x7efce0003ef0 2023-10-19T10:36:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80af510 con 0x7efce0004d00 2023-10-19T10:36:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce000b980 con 0x7efcf80b38f0 2023-10-19T10:36:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf6ffd640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce0003ef0 0x7efce00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf77fe640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 0x7efce0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf7fff640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0004d00 0x7efce0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf6ffd640 1 -- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 msgr2=0x7efce0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf6ffd640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf80b38f0 0x7efce0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf6ffd640 1 -- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0004d00 msgr2=0x7efce0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf6ffd640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0004d00 0x7efce0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.893+0000 7efcf6ffd640 1 -- 172.21.15.50:0/1333788206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0003520 con 0x7efce0003ef0 2023-10-19T10:36:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.894+0000 7efcf6ffd640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce0003ef0 0x7efce00047c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcf80b3490 tx=0x7efce00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d4876d68be8f48 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.894+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec005ad0 con 0x7efce0003ef0 2023-10-19T10:36:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.894+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec005c30 con 0x7efce0003ef0 2023-10-19T10:36:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.894+0000 7efcf67fc640 1 -- 172.21.15.50:0/1333788206 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec02c880 con 0x7efce0003ef0 2023-10-19T10:36:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.894+0000 7efcfdae3640 1 -- 172.21.15.50:0/1333788206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce0003ef0 msgr2=0x7efce00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.894+0000 7efcfdae3640 1 --2- 172.21.15.50:0/1333788206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce0003ef0 0x7efce00047c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcf80b3490 tx=0x7efce00057f0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.895+0000 7efcf7fff640 1 -- 172.21.15.50:0/1333788206 reap_dead start 2023-10-19T10:36:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.895+0000 7efcfdae3640 1 -- 172.21.15.50:0/1333788206 shutdown_connections 2023-10-19T10:36:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.895+0000 7efcfdae3640 1 -- 172.21.15.50:0/1333788206 >> 172.21.15.50:0/1333788206 conn(0x7efcf809b960 msgr2=0x7efcf80d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.895+0000 7efcfdae3640 1 -- 172.21.15.50:0/1333788206 shutdown_connections 2023-10-19T10:36:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.895+0000 7efcfdae3640 1 -- 172.21.15.50:0/1333788206 wait complete. 2023-10-19T10:36:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.896+0000 7efcfdae3640 1 Processor -- start 2023-10-19T10:36:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.896+0000 7efcfdae3640 1 -- start start 2023-10-19T10:36:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0003ef0 0x7efcf814a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf815d1e0 0x7efcf815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 0x7efcf80c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80ca4a0 con 0x7efcf80c9ae0 2023-10-19T10:36:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80b6ca0 con 0x7efce0003ef0 2023-10-19T10:36:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80b6090 con 0x7efcf815d1e0 2023-10-19T10:36:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 0x7efcf80c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 0x7efcf80c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60142/0 (socket says 172.21.15.50:60142) 2023-10-19T10:36:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 -- 172.21.15.50:0/3732366947 learned_addr learned my addr 172.21.15.50:0/3732366947 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf6ffd640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf815d1e0 0x7efcf815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf77fe640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0003ef0 0x7efcf814a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 -- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf815d1e0 msgr2=0x7efcf815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf815d1e0 0x7efcf815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 -- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0003ef0 msgr2=0x7efcf814a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0003ef0 0x7efcf814a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 -- 172.21.15.50:0/3732366947 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80b6370 con 0x7efcf80c9ae0 2023-10-19T10:36:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf77fe640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0003ef0 0x7efcf814a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcf7fff640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 0x7efcf80c9f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efcf8161620 tx=0x7efcf000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf000e040 con 0x7efcf80c9ae0 2023-10-19T10:36:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf000f040 con 0x7efcf80c9ae0 2023-10-19T10:36:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf80b6650 con 0x7efcf80c9ae0 2023-10-19T10:36:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0003bd0 con 0x7efcf80c9ae0 2023-10-19T10:36:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.897+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf814a750 con 0x7efcf80c9ae0 2023-10-19T10:36:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.898+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf80b4f40 con 0x7efcf80c9ae0 2023-10-19T10:36:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.899+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efcf0003d30 con 0x7efcf80c9ae0 2023-10-19T10:36:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.899+0000 7efcdffff640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efce406a050 0x7efce406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.899+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efcf0088a10 con 0x7efcf80c9ae0 2023-10-19T10:36:51.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.899+0000 7efcf77fe640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efce406a050 0x7efce406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:51.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.900+0000 7efcf77fe640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efce406a050 0x7efce406c510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efce8006ce0 tx=0x7efce8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:51.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:51.903+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7efcf0057ee0 con 0x7efcf80c9ae0 2023-10-19T10:36:52.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.077+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcf804e420 con 0x7efcf80c9ae0 2023-10-19T10:36:52.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.078+0000 7efcdffff640 1 -- 172.21.15.50:0/3732366947 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcf0057a70 con 0x7efcf80c9ae0 2023-10-19T10:36:52.081 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:36:52.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efce406a050 msgr2=0x7efce406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efce406a050 0x7efce406c510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efce8006ce0 tx=0x7efce8008040 comp rx=0 tx=0).stop 2023-10-19T10:36:52.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 msgr2=0x7efcf80c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 0x7efcf80c9f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efcf8161620 tx=0x7efcf000f580 comp rx=0 tx=0).stop 2023-10-19T10:36:52.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 shutdown_connections 2023-10-19T10:36:52.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efce406a050 0x7efce406c510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcf80c9ae0 0x7efcf80c9f60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcf815d1e0 0x7efcf815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 --2- 172.21.15.50:0/3732366947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce0003ef0 0x7efcf814a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 >> 172.21.15.50:0/3732366947 conn(0x7efcf809b960 msgr2=0x7efcf8072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 shutdown_connections 2023-10-19T10:36:52.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.080+0000 7efcfdae3640 1 -- 172.21.15.50:0/3732366947 wait complete. 2023-10-19T10:36:52.093 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:36:52.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.199+0000 7f3eb5cf5640 1 Processor -- start 2023-10-19T10:36:52.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.199+0000 7f3eb5cf5640 1 -- start start 2023-10-19T10:36:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.199+0000 7f3eb5cf5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b3090 0x7f3eb00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.199+0000 7f3eb5cf5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3eb0161210 0x7f3eb00b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.199+0000 7f3eb5cf5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00af250 con 0x7f3eb00b38f0 2023-10-19T10:36:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eb5cf5640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb00b0190 con 0x7f3eb0161210 2023-10-19T10:36:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eb5cf5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00af510 con 0x7f3eb00b3090 2023-10-19T10:36:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3eb0161210 0x7f3eb00b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3eb0161210 0x7f3eb00b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60156/0 (socket says 172.21.15.50:60156) 2023-10-19T10:36:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 -- 172.21.15.50:0/881534058 learned_addr learned my addr 172.21.15.50:0/881534058 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaeffd640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b3090 0x7f3eb00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 -- 172.21.15.50:0/881534058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b3090 msgr2=0x7f3eb00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b3090 0x7f3eb00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 -- 172.21.15.50:0/881534058 >> v1:172.21.15.183:6789/0 conn(0x7f3eb00b38f0 legacy=0x7f3eb0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaffff640 1 -- 172.21.15.50:0/881534058 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00af7f0 con 0x7f3eb0161210 2023-10-19T10:36:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.200+0000 7f3eaeffd640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b3090 0x7f3eb00d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eaffff640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0161210 0x7f3eb00b05b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009e50 tx=0x7f3ea802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9858914fdbf858f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eae7fc640 1 -- 172.21.15.50:0/881534058 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea802ca70 con 0x7f3eb0161210 2023-10-19T10:36:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eae7fc640 1 -- 172.21.15.50:0/881534058 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea802cbd0 con 0x7f3eb0161210 2023-10-19T10:36:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eae7fc640 1 -- 172.21.15.50:0/881534058 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80343f0 con 0x7f3eb0161210 2023-10-19T10:36:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/881534058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0161210 msgr2=0x7f3eb00b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0161210 0x7f3eb00b05b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009e50 tx=0x7f3ea802bed0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/881534058 shutdown_connections 2023-10-19T10:36:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0161210 0x7f3eb00b05b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/881534058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b3090 0x7f3eb00d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/881534058 >> 172.21.15.50:0/881534058 conn(0x7f3eb009b960 msgr2=0x7f3eb00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/881534058 shutdown_connections 2023-10-19T10:36:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.201+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/881534058 wait complete. 2023-10-19T10:36:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 Processor -- start 2023-10-19T10:36:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 -- start start 2023-10-19T10:36:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 0x7f3eb014abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b38f0 0x7f3eb00c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb0161210 0x7f3eb00cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c9800 con 0x7f3eb00b3090 2023-10-19T10:36:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00ca380 con 0x7f3eb0161210 2023-10-19T10:36:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.203+0000 7f3eb5cf5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c9a50 con 0x7f3eb00b38f0 2023-10-19T10:36:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 0x7f3eb014abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 0x7f3eb014abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60168/0 (socket says 172.21.15.50:60168) 2023-10-19T10:36:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 -- 172.21.15.50:0/2883019569 learned_addr learned my addr 172.21.15.50:0/2883019569 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaffff640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb0161210 0x7f3eb00cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaeffd640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b38f0 0x7f3eb00c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 -- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b38f0 msgr2=0x7f3eb00c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b38f0 0x7f3eb00c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 -- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb0161210 msgr2=0x7f3eb00cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb0161210 0x7f3eb00cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 -- 172.21.15.50:0/2883019569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009b30 con 0x7f3eb00b3090 2023-10-19T10:36:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eaf7fe640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 0x7f3eb014abf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c450 tx=0x7f3ea000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0011790 con 0x7f3eb00b3090 2023-10-19T10:36:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea00118f0 con 0x7f3eb00b3090 2023-10-19T10:36:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb00c9d30 con 0x7f3eb00b3090 2023-10-19T10:36:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0011ca0 con 0x7f3eb00b3090 2023-10-19T10:36:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.204+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb00cf630 con 0x7f3eb00b3090 2023-10-19T10:36:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.205+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb004e420 con 0x7f3eb00b3090 2023-10-19T10:36:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.206+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3ea0012420 con 0x7f3eb00b3090 2023-10-19T10:36:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.206+0000 7f3e97fff640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e9c06a050 0x7f3e9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.206+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3ea0087d00 con 0x7f3eb00b3090 2023-10-19T10:36:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.206+0000 7f3eaeffd640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e9c06a050 0x7f3e9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.207+0000 7f3eaeffd640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e9c06a050 0x7f3e9c06c510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0147640 tx=0x7f3ea4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.210+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3ea0053880 con 0x7f3eb00b3090 2023-10-19T10:36:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.388+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3eb00b86b0 con 0x7f3eb00b3090 2023-10-19T10:36:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.389+0000 7f3e97fff640 1 -- 172.21.15.50:0/2883019569 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ea0056ea0 con 0x7f3eb00b3090 2023-10-19T10:36:52.392 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:36:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e9c06a050 msgr2=0x7f3e9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e9c06a050 0x7f3e9c06c510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0147640 tx=0x7f3ea4008040 comp rx=0 tx=0).stop 2023-10-19T10:36:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 msgr2=0x7f3eb014abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 0x7f3eb014abf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c450 tx=0x7f3ea000c910 comp rx=0 tx=0).stop 2023-10-19T10:36:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 shutdown_connections 2023-10-19T10:36:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e9c06a050 0x7f3e9c06c510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb0161210 0x7f3eb00cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3eb00b38f0 0x7f3eb00c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 --2- 172.21.15.50:0/2883019569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb00b3090 0x7f3eb014abf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 >> 172.21.15.50:0/2883019569 conn(0x7f3eb009b960 msgr2=0x7f3eb00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 shutdown_connections 2023-10-19T10:36:52.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.391+0000 7f3eb5cf5640 1 -- 172.21.15.50:0/2883019569 wait complete. 2023-10-19T10:36:52.404 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-19T10:36:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.507+0000 7fbea24ee640 1 Processor -- start 2023-10-19T10:36:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.507+0000 7fbea24ee640 1 -- start start 2023-10-19T10:36:52.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbea24ee640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbe9c15c570 0x7fbe9c0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbea24ee640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c06aa30 con 0x7fbe9c15c950 2023-10-19T10:36:52.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbea24ee640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c06b2d0 con 0x7fbe9c0cd920 2023-10-19T10:36:52.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbea24ee640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0ce090 con 0x7fbe9c15c570 2023-10-19T10:36:52.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbe9b7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fbe9c15c950 0x7fbe9c06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48922/0 (socket says 172.21.15.50:48922) 2023-10-19T10:36:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbe9b7fe640 1 -- 172.21.15.50:0/629364396 learned_addr learned my addr 172.21.15.50:0/629364396 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbe9bfff640 1 --2- 172.21.15.50:0/629364396 >> v2:172.21.15.183:3300/0 conn(0x7fbe9c15c570 0x7fbe9c0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbe9affd640 1 -- 172.21.15.50:0/629364396 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479984164 0 0) 0x7fbe9c06aa30 con 0x7fbe9c15c950 2023-10-19T10:36:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.508+0000 7fbe9affd640 1 -- 172.21.15.50:0/629364396 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe88003520 con 0x7fbe9c15c950 2023-10-19T10:36:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.509+0000 7fbe9bfff640 1 -- 172.21.15.50:0/629364396 >> v1:172.21.15.183:6789/0 conn(0x7fbe9c0cd920 legacy=0x7fbe9c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.509+0000 7fbe9bfff640 1 -- 172.21.15.50:0/629364396 >> v1:172.21.15.50:6789/0 conn(0x7fbe9c15c950 legacy=0x7fbe9c06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.509+0000 7fbe9affd640 1 -- 172.21.15.50:0/629364396 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1316578145 0 0) 0x7fbe88003520 con 0x7fbe9c15c950 2023-10-19T10:36:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.509+0000 7fbe9bfff640 1 -- 172.21.15.50:0/629364396 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c06ac80 con 0x7fbe9c15c570 2023-10-19T10:36:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.509+0000 7fbe9bfff640 1 --2- 172.21.15.50:0/629364396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0d0e30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009790 tx=0x7fbe8402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b871cabd8c1faa server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbe9affd640 1 -- 172.21.15.50:0/629364396 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c001670 con 0x7fbe9c15c570 2023-10-19T10:36:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbe9affd640 1 -- 172.21.15.50:0/629364396 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8402e050 con 0x7fbe9c15c570 2023-10-19T10:36:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbe9affd640 1 -- 172.21.15.50:0/629364396 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84030730 con 0x7fbe9c15c570 2023-10-19T10:36:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 -- 172.21.15.50:0/629364396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c15c570 msgr2=0x7fbe9c0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 --2- 172.21.15.50:0/629364396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0d0e30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009790 tx=0x7fbe8402b790 comp rx=0 tx=0).stop 2023-10-19T10:36:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 -- 172.21.15.50:0/629364396 shutdown_connections 2023-10-19T10:36:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 --2- 172.21.15.50:0/629364396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0d0e30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 -- 172.21.15.50:0/629364396 >> 172.21.15.50:0/629364396 conn(0x7fbe9c089ff0 msgr2=0x7fbe9c09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 -- 172.21.15.50:0/629364396 shutdown_connections 2023-10-19T10:36:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.510+0000 7fbea24ee640 1 -- 172.21.15.50:0/629364396 wait complete. 2023-10-19T10:36:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.511+0000 7fbea24ee640 1 Processor -- start 2023-10-19T10:36:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.511+0000 7fbea24ee640 1 -- start start 2023-10-19T10:36:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea24ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c0cd920 0x7fbe9c0c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea24ee640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea24ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe9c15c950 0x7fbe9c0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea24ee640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b2170 con 0x7fbe9c15c570 2023-10-19T10:36:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea24ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0c9080 con 0x7fbe9c0cd920 2023-10-19T10:36:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea24ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0c9360 con 0x7fbe9c15c950 2023-10-19T10:36:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbe9b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbe9b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60170/0 (socket says 172.21.15.50:60170) 2023-10-19T10:36:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbe9b7fe640 1 -- 172.21.15.50:0/895961493 learned_addr learned my addr 172.21.15.50:0/895961493 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.512+0000 7fbea0a8b640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe9c15c950 0x7fbe9c0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe9b7fe640 1 -- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe9c15c950 msgr2=0x7fbe9c0c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe9b7fe640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe9c15c950 0x7fbe9c0c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe9b7fe640 1 -- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c0cd920 msgr2=0x7fbe9c0c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe9b7fe640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c0cd920 0x7fbe9c0c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe9b7fe640 1 -- 172.21.15.50:0/895961493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840093a0 con 0x7fbe9c15c570 2023-10-19T10:36:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe9b7fe640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0c8270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c730 tx=0x7fbe8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c005ad0 con 0x7fbe9c15c570 2023-10-19T10:36:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c005c50 con 0x7fbe9c15c570 2023-10-19T10:36:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c0b23c0 con 0x7fbe9c15c570 2023-10-19T10:36:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c007070 con 0x7fbe9c15c570 2023-10-19T10:36:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.513+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c0c8cf0 con 0x7fbe9c15c570 2023-10-19T10:36:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.514+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c0c9520 con 0x7fbe9c15c570 2023-10-19T10:36:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.514+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbe8c0071d0 con 0x7fbe9c15c570 2023-10-19T10:36:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.515+0000 7fbe98ff9640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe7c06a050 0x7fbe7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.515+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbe8c087a60 con 0x7fbe9c15c570 2023-10-19T10:36:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.515+0000 7fbe9bfff640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe7c06a050 0x7fbe7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.516+0000 7fbe9bfff640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe7c06a050 0x7fbe7c06c510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe84034000 tx=0x7fbe8402d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.519+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbe8c053410 con 0x7fbe9c15c570 2023-10-19T10:36:52.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.697+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbe9c073990 con 0x7fbe9c15c570 2023-10-19T10:36:52.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.697+0000 7fbe98ff9640 1 -- 172.21.15.50:0/895961493 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbe8c017090 con 0x7fbe9c15c570 2023-10-19T10:36:52.700 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:36:52.700 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2023-10-19T10:36:52.701 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:36:52.701 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2023-10-19T10:36:52.701 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2023-10-19T10:36:52.701 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:36:52.701 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:36:52.702 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 2, 2023-10-19T10:36:52.702 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": "rbd" 2023-10-19T10:36:52.702 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:36:52.702 INFO:teuthology.orchestra.run.smithi050.stdout:] 2023-10-19T10:36:52.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe7c06a050 msgr2=0x7fbe7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe7c06a050 0x7fbe7c06c510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe84034000 tx=0x7fbe8402d000 comp rx=0 tx=0).stop 2023-10-19T10:36:52.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 msgr2=0x7fbe9c0c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0c8270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c730 tx=0x7fbe8c00cbf0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 shutdown_connections 2023-10-19T10:36:52.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe7c06a050 0x7fbe7c06c510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe9c15c950 0x7fbe9c0c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c15c570 0x7fbe9c0c8270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 --2- 172.21.15.50:0/895961493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe9c0cd920 0x7fbe9c0c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 >> 172.21.15.50:0/895961493 conn(0x7fbe9c089ff0 msgr2=0x7fbe9c15bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:52.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 shutdown_connections 2023-10-19T10:36:52.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.699+0000 7fbea24ee640 1 -- 172.21.15.50:0/895961493 wait complete. 2023-10-19T10:36:52.712 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-19T10:36:52.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-19T10:36:52.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e54feb640 1 Processor -- start 2023-10-19T10:36:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e54feb640 1 -- start start 2023-10-19T10:36:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e54feb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e5015c620 0x7f2e500b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e54feb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e500cda30 con 0x7f2e500cfb00 2023-10-19T10:36:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e54feb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e50161110 con 0x7f2e500b82f0 2023-10-19T10:36:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e54feb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500ccc10 con 0x7f2e5015c620 2023-10-19T10:36:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.821+0000 7f2e4ed9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2e500cfb00 0x7f2e500d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48936/0 (socket says 172.21.15.50:48936) 2023-10-19T10:36:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4ed9d640 1 -- 172.21.15.50:0/3127590443 learned_addr learned my addr 172.21.15.50:0/3127590443 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4e59c640 1 --2- 172.21.15.50:0/3127590443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e5015c620 0x7f2e500b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012957235 0 0) 0x7f2e500cda30 con 0x7f2e500cfb00 2023-10-19T10:36:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e3c003520 con 0x7f2e500cfb00 2023-10-19T10:36:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590607440 0 0) 0x7f2e50161110 con 0x7f2e500b82f0 2023-10-19T10:36:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e500cda30 con 0x7f2e500b82f0 2023-10-19T10:36:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4e59c640 1 -- 172.21.15.50:0/3127590443 >> v1:172.21.15.183:6789/0 conn(0x7f2e500b82f0 legacy=0x7f2e500cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4e59c640 1 -- 172.21.15.50:0/3127590443 >> v1:172.21.15.50:6789/0 conn(0x7f2e500cfb00 legacy=0x7f2e500d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.822+0000 7f2e4e59c640 1 -- 172.21.15.50:0/3127590443 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500cce20 con 0x7f2e5015c620 2023-10-19T10:36:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e4e59c640 1 --2- 172.21.15.50:0/3127590443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e5015c620 0x7f2e500b7db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e400097d0 tx=0x7f2e4002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161a13393b605372 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40031890 con 0x7f2e5015c620 2023-10-19T10:36:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e400319f0 con 0x7f2e5015c620 2023-10-19T10:36:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e4dd9b640 1 -- 172.21.15.50:0/3127590443 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40031d60 con 0x7f2e5015c620 2023-10-19T10:36:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e54feb640 1 -- 172.21.15.50:0/3127590443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e5015c620 msgr2=0x7f2e500b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e54feb640 1 --2- 172.21.15.50:0/3127590443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e5015c620 0x7f2e500b7db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e400097d0 tx=0x7f2e4002b930 comp rx=0 tx=0).stop 2023-10-19T10:36:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e54feb640 1 -- 172.21.15.50:0/3127590443 shutdown_connections 2023-10-19T10:36:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e54feb640 1 --2- 172.21.15.50:0/3127590443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e5015c620 0x7f2e500b7db0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.823+0000 7f2e54feb640 1 -- 172.21.15.50:0/3127590443 >> 172.21.15.50:0/3127590443 conn(0x7f2e50071110 msgr2=0x7f2e500714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.824+0000 7f2e54feb640 1 -- 172.21.15.50:0/3127590443 shutdown_connections 2023-10-19T10:36:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.824+0000 7f2e54feb640 1 -- 172.21.15.50:0/3127590443 wait complete. 2023-10-19T10:36:52.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.825+0000 7f2e54feb640 1 Processor -- start 2023-10-19T10:36:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.825+0000 7f2e54feb640 1 -- start start 2023-10-19T10:36:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e500b82f0 0x7f2e5015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e500cfb00 0x7f2e500b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 0x7f2e500b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e501508d0 con 0x7f2e5015c620 2023-10-19T10:36:52.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5015efc0 con 0x7f2e500b82f0 2023-10-19T10:36:52.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5015fa90 con 0x7f2e500cfb00 2023-10-19T10:36:52.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 0x7f2e500b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 0x7f2e500b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60182/0 (socket says 172.21.15.50:60182) 2023-10-19T10:36:52.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 -- 172.21.15.50:0/2288064714 learned_addr learned my addr 172.21.15.50:0/2288064714 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:52.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4e59c640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e500b82f0 0x7f2e5015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e47fff640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e500cfb00 0x7f2e500b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 -- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e500cfb00 msgr2=0x7f2e500b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e500cfb00 0x7f2e500b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 -- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e500b82f0 msgr2=0x7f2e5015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:52.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e500b82f0 0x7f2e5015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:52.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 -- 172.21.15.50:0/2288064714 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e40009400 con 0x7f2e5015c620 2023-10-19T10:36:52.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e47fff640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e500cfb00 0x7f2e500b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:52.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e4ed9d640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 0x7f2e500b2a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800cbc0 tx=0x7f2e4800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800ace0 con 0x7f2e5015c620 2023-10-19T10:36:52.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4800ae40 con 0x7f2e5015c620 2023-10-19T10:36:52.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5015f1f0 con 0x7f2e5015c620 2023-10-19T10:36:52.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48012600 con 0x7f2e5015c620 2023-10-19T10:36:52.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.826+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5015f7b0 con 0x7f2e5015c620 2023-10-19T10:36:52.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.827+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5004e420 con 0x7f2e5015c620 2023-10-19T10:36:52.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.828+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e48012760 con 0x7f2e5015c620 2023-10-19T10:36:52.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.828+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3006a020 0x7f2e3006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:52.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.828+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2e48088060 con 0x7f2e5015c620 2023-10-19T10:36:52.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.829+0000 7f2e4e59c640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3006a020 0x7f2e3006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:52.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.829+0000 7f2e4e59c640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3006a020 0x7f2e3006c4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e40036000 tx=0x7f2e4003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:52.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:52.833+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2e48053be0 con 0x7f2e5015c620 2023-10-19T10:36:53.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.006+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2e50154a80 con 0x7f2e5015c620 2023-10-19T10:36:53.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.286+0000 7f2e46ffd640 1 -- 172.21.15.50:0/2288064714 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2e48057200 con 0x7f2e5015c620 2023-10-19T10:36:53.289 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2023-10-19T10:36:53.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3006a020 msgr2=0x7f2e3006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:53.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3006a020 0x7f2e3006c4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e40036000 tx=0x7f2e4003c040 comp rx=0 tx=0).stop 2023-10-19T10:36:53.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 msgr2=0x7f2e500b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 0x7f2e500b2a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800cbc0 tx=0x7f2e4800a430 comp rx=0 tx=0).stop 2023-10-19T10:36:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 shutdown_connections 2023-10-19T10:36:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3006a020 0x7f2e3006c4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e5015c620 0x7f2e500b2a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e500cfb00 0x7f2e500b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 --2- 172.21.15.50:0/2288064714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e500b82f0 0x7f2e5015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 >> 172.21.15.50:0/2288064714 conn(0x7f2e50071110 msgr2=0x7f2e5015ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 shutdown_connections 2023-10-19T10:36:53.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.291+0000 7f2e54feb640 1 -- 172.21.15.50:0/2288064714 wait complete. 2023-10-19T10:36:53.305 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-19T10:36:53.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.417+0000 7fc9addf7640 1 Processor -- start 2023-10-19T10:36:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.417+0000 7fc9addf7640 1 -- start start 2023-10-19T10:36:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9addf7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9a80b38f0 0x7fc9a80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9addf7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a8161210 0x7fc9a80b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9addf7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80af250 con 0x7fc9a80b3090 2023-10-19T10:36:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9addf7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a80b0190 con 0x7fc9a80b38f0 2023-10-19T10:36:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9addf7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80af510 con 0x7fc9a8161210 2023-10-19T10:36:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9a80b38f0 0x7fc9a80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9a80b38f0 0x7fc9a80d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60188/0 (socket says 172.21.15.50:60188) 2023-10-19T10:36:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 -- 172.21.15.50:0/1394103628 learned_addr learned my addr 172.21.15.50:0/1394103628 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a7fff640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a8161210 0x7fc9a80b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 -- 172.21.15.50:0/1394103628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a8161210 msgr2=0x7fc9a80b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a8161210 0x7fc9a80b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 -- 172.21.15.50:0/1394103628 >> v1:172.21.15.183:6789/0 conn(0x7fc9a80b3090 legacy=0x7fc9a8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a77fe640 1 -- 172.21.15.50:0/1394103628 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80af7f0 con 0x7fc9a80b38f0 2023-10-19T10:36:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.418+0000 7fc9a7fff640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a8161210 0x7fc9a80b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:36:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.419+0000 7fc9a77fe640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80d30e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc998009850 tx=0x7fc99802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a17b5e4ad1d06b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.419+0000 7fc9a67fc640 1 -- 172.21.15.50:0/1394103628 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998004230 con 0x7fc9a80b38f0 2023-10-19T10:36:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.419+0000 7fc9a67fc640 1 -- 172.21.15.50:0/1394103628 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc998004390 con 0x7fc9a80b38f0 2023-10-19T10:36:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.419+0000 7fc9a67fc640 1 -- 172.21.15.50:0/1394103628 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998031830 con 0x7fc9a80b38f0 2023-10-19T10:36:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.419+0000 7fc9addf7640 1 -- 172.21.15.50:0/1394103628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a80b38f0 msgr2=0x7fc9a80d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.419+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80d30e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc998009850 tx=0x7fc99802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.420+0000 7fc9addf7640 1 -- 172.21.15.50:0/1394103628 shutdown_connections 2023-10-19T10:36:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.420+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a8161210 0x7fc9a80b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.420+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1394103628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80d30e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.420+0000 7fc9addf7640 1 -- 172.21.15.50:0/1394103628 >> 172.21.15.50:0/1394103628 conn(0x7fc9a809b960 msgr2=0x7fc9a80d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:53.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.420+0000 7fc9addf7640 1 -- 172.21.15.50:0/1394103628 shutdown_connections 2023-10-19T10:36:53.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.420+0000 7fc9addf7640 1 -- 172.21.15.50:0/1394103628 wait complete. 2023-10-19T10:36:53.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.421+0000 7fc9addf7640 1 Processor -- start 2023-10-19T10:36:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.421+0000 7fc9addf7640 1 -- start start 2023-10-19T10:36:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a80b3090 0x7fc9a8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 0x7fc9a80c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8158420 con 0x7fc9a8161210 2023-10-19T10:36:53.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80b6060 con 0x7fc9a80b38f0 2023-10-19T10:36:53.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80b6340 con 0x7fc9a80b3090 2023-10-19T10:36:53.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 0x7fc9a80c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:53.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 0x7fc9a80c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60196/0 (socket says 172.21.15.50:60196) 2023-10-19T10:36:53.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 -- 172.21.15.50:0/1543970931 learned_addr learned my addr 172.21.15.50:0/1543970931 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a6ffd640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a77fe640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a80b3090 0x7fc9a8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 -- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a80b3090 msgr2=0x7fc9a8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a80b3090 0x7fc9a8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 -- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a80b38f0 msgr2=0x7fc9a80cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 -- 172.21.15.50:0/1543970931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998009460 con 0x7fc9a8161210 2023-10-19T10:36:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9a7fff640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 0x7fc9a80c6ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000ccf0 tx=0x7fc9a000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a000f680 con 0x7fc9a8161210 2023-10-19T10:36:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a000f800 con 0x7fc9a8161210 2023-10-19T10:36:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a000fbb0 con 0x7fc9a8161210 2023-10-19T10:36:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.422+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a80b6620 con 0x7fc9a8161210 2023-10-19T10:36:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.423+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a80c71e0 con 0x7fc9a8161210 2023-10-19T10:36:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.423+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a80b5ec0 con 0x7fc9a8161210 2023-10-19T10:36:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.424+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9a000b040 con 0x7fc9a8161210 2023-10-19T10:36:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.425+0000 7fc98ffff640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc99406a050 0x7fc99406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.425+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fc9a0088dc0 con 0x7fc9a8161210 2023-10-19T10:36:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.425+0000 7fc9a77fe640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc99406a050 0x7fc99406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.425+0000 7fc9a77fe640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc99406a050 0x7fc99406c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc998035000 tx=0x7fc998008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.429+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc9a80b5ec0 con 0x7fc9a8161210 2023-10-19T10:36:53.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:53.603+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc9a80b5ec0 con 0x7fc9a8161210 2023-10-19T10:36:54.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.288+0000 7fc98ffff640 1 -- 172.21.15.50:0/1543970931 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc9a80b5ec0 con 0x7fc9a8161210 2023-10-19T10:36:54.292 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2023-10-19T10:36:54.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.297+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc99406a050 msgr2=0x7fc99406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:54.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.297+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc99406a050 0x7fc99406c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc998035000 tx=0x7fc998008030 comp rx=0 tx=0).stop 2023-10-19T10:36:54.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.297+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 msgr2=0x7fc9a80c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:54.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.297+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 0x7fc9a80c6ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000ccf0 tx=0x7fc9a000a650 comp rx=0 tx=0).stop 2023-10-19T10:36:54.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.303+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 shutdown_connections 2023-10-19T10:36:54.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.303+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc99406a050 0x7fc99406c510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.303+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8161210 0x7fc9a80c6ca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.303+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a80b38f0 0x7fc9a80cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.303+0000 7fc9addf7640 1 --2- 172.21.15.50:0/1543970931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9a80b3090 0x7fc9a8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.303+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 >> 172.21.15.50:0/1543970931 conn(0x7fc9a809b960 msgr2=0x7fc9a80d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:54.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.308+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 shutdown_connections 2023-10-19T10:36:54.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.309+0000 7fc9addf7640 1 -- 172.21.15.50:0/1543970931 wait complete. 2023-10-19T10:36:54.325 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-19T10:36:54.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.435+0000 7f9f951fe640 1 Processor -- start 2023-10-19T10:36:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.435+0000 7f9f951fe640 1 -- start start 2023-10-19T10:36:54.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f951fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f9015c760 0x7f9f900d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:54.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f951fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:54.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f951fe640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9006bc10 con 0x7f9f900d2850 2023-10-19T10:36:54.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f951fe640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9006c790 con 0x7f9f9015c760 2023-10-19T10:36:54.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f951fe640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9006bef0 con 0x7f9f9006b670 2023-10-19T10:36:54.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f9015c760 0x7f9f900d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:54.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f9015c760 0x7f9f900d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60204/0 (socket says 172.21.15.50:60204) 2023-10-19T10:36:54.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 -- 172.21.15.50:0/141535550 learned_addr learned my addr 172.21.15.50:0/141535550 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:54.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8e59c640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:54.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 -- 172.21.15.50:0/141535550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 msgr2=0x7f9f900d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:54.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 -- 172.21.15.50:0/141535550 >> v1:172.21.15.183:6789/0 conn(0x7f9f900d2850 legacy=0x7f9f901602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:54.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8ed9d640 1 -- 172.21.15.50:0/141535550 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9006c1d0 con 0x7f9f9015c760 2023-10-19T10:36:54.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.436+0000 7f9f8e59c640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:54.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.437+0000 7f9f8ed9d640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f9015c760 0x7f9f900d0ce0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009e50 tx=0x7f9f8002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6357d63e2a075611 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:54.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.437+0000 7f9f8dd9b640 1 -- 172.21.15.50:0/141535550 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8002cb00 con 0x7f9f9015c760 2023-10-19T10:36:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.437+0000 7f9f8dd9b640 1 -- 172.21.15.50:0/141535550 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8002cc60 con 0x7f9f9015c760 2023-10-19T10:36:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.437+0000 7f9f8dd9b640 1 -- 172.21.15.50:0/141535550 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800343f0 con 0x7f9f9015c760 2023-10-19T10:36:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 -- 172.21.15.50:0/141535550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f9015c760 msgr2=0x7f9f900d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f9015c760 0x7f9f900d0ce0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009e50 tx=0x7f9f8002bf60 comp rx=0 tx=0).stop 2023-10-19T10:36:54.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 -- 172.21.15.50:0/141535550 shutdown_connections 2023-10-19T10:36:54.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 --2- 172.21.15.50:0/141535550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f9015c760 0x7f9f900d0ce0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 -- 172.21.15.50:0/141535550 >> 172.21.15.50:0/141535550 conn(0x7f9f90089ff0 msgr2=0x7f9f9009b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:54.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 -- 172.21.15.50:0/141535550 shutdown_connections 2023-10-19T10:36:54.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.438+0000 7f9f951fe640 1 -- 172.21.15.50:0/141535550 wait complete. 2023-10-19T10:36:54.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.439+0000 7f9f951fe640 1 Processor -- start 2023-10-19T10:36:54.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.439+0000 7f9f951fe640 1 -- start start 2023-10-19T10:36:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f951fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f951fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 0x7f9f900b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f951fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f9015c760 0x7f9f901f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f951fe640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900cef70 con 0x7f9f900d2850 2023-10-19T10:36:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f951fe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900cf940 con 0x7f9f9015c760 2023-10-19T10:36:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f951fe640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900b80e0 con 0x7f9f9006b670 2023-10-19T10:36:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 0x7f9f900b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 0x7f9f900b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60206/0 (socket says 172.21.15.50:60206) 2023-10-19T10:36:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 -- 172.21.15.50:0/700443257 learned_addr learned my addr 172.21.15.50:0/700443257 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8f59e640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f9015c760 0x7f9f901f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8ed9d640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 -- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 msgr2=0x7f9f900cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 -- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f9015c760 msgr2=0x7f9f901f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f9015c760 0x7f9f901f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:54.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 -- 172.21.15.50:0/700443257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f80009b30 con 0x7f9f900d2850 2023-10-19T10:36:54.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.440+0000 7f9f8e59c640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 0x7f9f900b5290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400e7e0 tx=0x7f9f8400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:54.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.441+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84004030 con 0x7f9f900d2850 2023-10-19T10:36:54.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.441+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f84004190 con 0x7f9f900d2850 2023-10-19T10:36:54.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.441+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f900b83c0 con 0x7f9f900d2850 2023-10-19T10:36:54.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.441+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84004520 con 0x7f9f900d2850 2023-10-19T10:36:54.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.441+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f900cec00 con 0x7f9f900d2850 2023-10-19T10:36:54.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.442+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f900cf0e0 con 0x7f9f900d2850 2023-10-19T10:36:54.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.442+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f8400c6d0 con 0x7f9f900d2850 2023-10-19T10:36:54.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.443+0000 7f9f777fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f7c06a050 0x7f9f7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:54.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.443+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f9f84088220 con 0x7f9f900d2850 2023-10-19T10:36:54.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.443+0000 7f9f8ed9d640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f7c06a050 0x7f9f7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:54.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.443+0000 7f9f8ed9d640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f7c06a050 0x7f9f7c06c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f80035000 tx=0x7f9f80030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:54.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.447+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9f84053980 con 0x7f9f900d2850 2023-10-19T10:36:54.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:54.659+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9f9004e420 con 0x7f9f900d2850 2023-10-19T10:36:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.301+0000 7f9f777fe640 1 -- 172.21.15.50:0/700443257 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9f84056fa0 con 0x7f9f900d2850 2023-10-19T10:36:55.304 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 3 and data pool 4 2023-10-19T10:36:55.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f7c06a050 msgr2=0x7f9f7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:55.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f7c06a050 0x7f9f7c06c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f80035000 tx=0x7f9f80030000 comp rx=0 tx=0).stop 2023-10-19T10:36:55.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 msgr2=0x7f9f900b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:55.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 0x7f9f900b5290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400e7e0 tx=0x7f9f8400eca0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 shutdown_connections 2023-10-19T10:36:55.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f7c06a050 0x7f9f7c06c510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f9015c760 0x7f9f901f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f900d2850 0x7f9f900b5290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 --2- 172.21.15.50:0/700443257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f9006b670 0x7f9f900cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.304+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 >> 172.21.15.50:0/700443257 conn(0x7f9f90089ff0 msgr2=0x7f9f900cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.305+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 shutdown_connections 2023-10-19T10:36:55.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.305+0000 7f9f951fe640 1 -- 172.21.15.50:0/700443257 wait complete. 2023-10-19T10:36:55.325 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-19T10:36:55.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-19T10:36:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 Processor -- start 2023-10-19T10:36:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 -- start start 2023-10-19T10:36:55.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa82c15c620 0x7fa82c0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:55.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 0x7fa82c0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:55.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0cda30 con 0x7fa82c0b82f0 2023-10-19T10:36:55.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0ccc10 con 0x7fa82c15c620 2023-10-19T10:36:55.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.441+0000 7fa831db2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c161110 con 0x7fa82c0cfb00 2023-10-19T10:36:55.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82b7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa82c15c620 0x7fa82c0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:55.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 0x7fa82c0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:55.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 0x7fa82c0d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:37620/0 (socket says 172.21.15.50:37620) 2023-10-19T10:36:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 -- 172.21.15.50:0/4101470052 learned_addr learned my addr 172.21.15.50:0/4101470052 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 -- 172.21.15.50:0/4101470052 >> v2:172.21.15.183:3300/0 conn(0x7fa82c15c620 msgr2=0x7fa82c0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 --2- 172.21.15.50:0/4101470052 >> v2:172.21.15.183:3300/0 conn(0x7fa82c15c620 0x7fa82c0b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 -- 172.21.15.50:0/4101470052 >> v1:172.21.15.183:6789/0 conn(0x7fa82c0b82f0 legacy=0x7fa82c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:55.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82bfff640 1 -- 172.21.15.50:0/4101470052 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0cce20 con 0x7fa82c0cfb00 2023-10-19T10:36:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.442+0000 7fa82b7fe640 1 --2- 172.21.15.50:0/4101470052 >> v2:172.21.15.183:3300/0 conn(0x7fa82c15c620 0x7fa82c0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:36:55.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.443+0000 7fa82bfff640 1 --2- 172.21.15.50:0/4101470052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 0x7fa82c0d0b00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa824009e50 tx=0x7fa82402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ffcfaf3b393bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:55.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.443+0000 7fa82affd640 1 -- 172.21.15.50:0/4101470052 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82402cb00 con 0x7fa82c0cfb00 2023-10-19T10:36:55.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.443+0000 7fa82affd640 1 -- 172.21.15.50:0/4101470052 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82402cc60 con 0x7fa82c0cfb00 2023-10-19T10:36:55.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.443+0000 7fa82affd640 1 -- 172.21.15.50:0/4101470052 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240343f0 con 0x7fa82c0cfb00 2023-10-19T10:36:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 -- 172.21.15.50:0/4101470052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 msgr2=0x7fa82c0d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 --2- 172.21.15.50:0/4101470052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 0x7fa82c0d0b00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa824009e50 tx=0x7fa82402bf60 comp rx=0 tx=0).stop 2023-10-19T10:36:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 -- 172.21.15.50:0/4101470052 shutdown_connections 2023-10-19T10:36:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 --2- 172.21.15.50:0/4101470052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0cfb00 0x7fa82c0d0b00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 --2- 172.21.15.50:0/4101470052 >> v2:172.21.15.183:3300/0 conn(0x7fa82c15c620 0x7fa82c0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 -- 172.21.15.50:0/4101470052 >> 172.21.15.50:0/4101470052 conn(0x7fa82c071110 msgr2=0x7fa82c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 -- 172.21.15.50:0/4101470052 shutdown_connections 2023-10-19T10:36:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.444+0000 7fa831db2640 1 -- 172.21.15.50:0/4101470052 wait complete. 2023-10-19T10:36:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.445+0000 7fa831db2640 1 Processor -- start 2023-10-19T10:36:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.445+0000 7fa831db2640 1 -- start start 2023-10-19T10:36:55.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa831db2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0b82f0 0x7fa82c1f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:55.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa831db2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82c0cfb00 0x7fa82c0b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:55.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa831db2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 0x7fa82c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:55.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa831db2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15ecb0 con 0x7fa82c15c620 2023-10-19T10:36:55.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa831db2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15ef90 con 0x7fa82c0cfb00 2023-10-19T10:36:55.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa831db2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15f270 con 0x7fa82c0b82f0 2023-10-19T10:36:55.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 0x7fa82c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:55.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 0x7fa82c0b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60214/0 (socket says 172.21.15.50:60214) 2023-10-19T10:36:55.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 -- 172.21.15.50:0/3920236621 learned_addr learned my addr 172.21.15.50:0/3920236621 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:55.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82b7fe640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0b82f0 0x7fa82c1f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:55.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa822ffd640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82c0cfb00 0x7fa82c0b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:55.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 -- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0b82f0 msgr2=0x7fa82c1f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:55.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0b82f0 0x7fa82c1f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 -- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82c0cfb00 msgr2=0x7fa82c0b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:55.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82c0cfb00 0x7fa82c0b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:55.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.446+0000 7fa82bfff640 1 -- 172.21.15.50:0/3920236621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c009400 con 0x7fa82c15c620 2023-10-19T10:36:55.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.447+0000 7fa82bfff640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 0x7fa82c0b3e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa824009e50 tx=0x7fa824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:55.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.447+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824009510 con 0x7fa82c15c620 2023-10-19T10:36:55.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.447+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82403e450 con 0x7fa82c15c620 2023-10-19T10:36:55.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.447+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa824009b30 con 0x7fa82c15c620 2023-10-19T10:36:55.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.447+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82403e800 con 0x7fa82c15c620 2023-10-19T10:36:55.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.447+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c15f830 con 0x7fa82c15c620 2023-10-19T10:36:55.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.448+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c15fa20 con 0x7fa82c15c620 2023-10-19T10:36:55.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.448+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa82403e960 con 0x7fa82c15c620 2023-10-19T10:36:55.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.449+0000 7fa828ff9640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa80c06a050 0x7fa80c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:55.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.449+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8240ab1b0 con 0x7fa82c15c620 2023-10-19T10:36:55.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.449+0000 7fa82b7fe640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa80c06a050 0x7fa80c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:55.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.449+0000 7fa82b7fe640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa80c06a050 0x7fa80c06c510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009cd0 tx=0x7fa81c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:55.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.453+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa824076950 con 0x7fa82c15c620 2023-10-19T10:36:55.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:55.640+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa82c04e420 con 0x7fa82c15c620 2023-10-19T10:36:55.715 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:36:55.718 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:36:56.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.096+0000 7fa828ff9640 1 -- 172.21.15.50:0/3920236621 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa824079f70 con 0x7fa82c15c620 2023-10-19T10:36:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa80c06a050 msgr2=0x7fa80c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa80c06a050 0x7fa80c06c510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009cd0 tx=0x7fa81c009190 comp rx=0 tx=0).stop 2023-10-19T10:36:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 msgr2=0x7fa82c0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 0x7fa82c0b3e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa824009e50 tx=0x7fa824004a40 comp rx=0 tx=0).stop 2023-10-19T10:36:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 shutdown_connections 2023-10-19T10:36:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa80c06a050 0x7fa80c06c510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa82c15c620 0x7fa82c0b3e00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82c0cfb00 0x7fa82c0b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 --2- 172.21.15.50:0/3920236621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa82c0b82f0 0x7fa82c1f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 >> 172.21.15.50:0/3920236621 conn(0x7fa82c071110 msgr2=0x7fa82c15af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:56.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 shutdown_connections 2023-10-19T10:36:56.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.099+0000 7fa831db2640 1 -- 172.21.15.50:0/3920236621 wait complete. 2023-10-19T10:36:56.114 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-19T10:36:56.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.226+0000 7f0292841640 1 Processor -- start 2023-10-19T10:36:56.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.226+0000 7f0292841640 1 -- start start 2023-10-19T10:36:56.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f0292841640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:56.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f0292841640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f028c0d04c0 0x7f028c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:56.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f0292841640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c15d930 con 0x7f028c0b1540 2023-10-19T10:36:56.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f0292841640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c0d0900 con 0x7f028c0d04c0 2023-10-19T10:36:56.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f0292841640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0cfc90 con 0x7f028c0cf370 2023-10-19T10:36:56.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f028c0d04c0 0x7f028c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:56.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f028c0d04c0 0x7f028c0cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60218/0 (socket says 172.21.15.50:60218) 2023-10-19T10:36:56.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 -- 172.21.15.50:0/1455568357 learned_addr learned my addr 172.21.15.50:0/1455568357 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:56.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028bfff640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:56.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 -- 172.21.15.50:0/1455568357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 msgr2=0x7f028c15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:56.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 -- 172.21.15.50:0/1455568357 >> v1:172.21.15.183:6789/0 conn(0x7f028c0b1540 legacy=0x7f028c1602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:56.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028b7fe640 1 -- 172.21.15.50:0/1455568357 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c15cd20 con 0x7f028c0d04c0 2023-10-19T10:36:56.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.227+0000 7f028bfff640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:56.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.228+0000 7f028b7fe640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0d04c0 0x7f028c0cf750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0280009870 tx=0x7f028002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc26b85573887b86 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:56.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.228+0000 7f028affd640 1 -- 172.21.15.50:0/1455568357 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028002f840 con 0x7f028c0d04c0 2023-10-19T10:36:56.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f028affd640 1 -- 172.21.15.50:0/1455568357 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028002f9a0 con 0x7f028c0d04c0 2023-10-19T10:36:56.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f028affd640 1 -- 172.21.15.50:0/1455568357 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028002fd30 con 0x7f028c0d04c0 2023-10-19T10:36:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 -- 172.21.15.50:0/1455568357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0d04c0 msgr2=0x7f028c0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0d04c0 0x7f028c0cf750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0280009870 tx=0x7f028002ba20 comp rx=0 tx=0).stop 2023-10-19T10:36:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 -- 172.21.15.50:0/1455568357 shutdown_connections 2023-10-19T10:36:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0d04c0 0x7f028c0cf750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 --2- 172.21.15.50:0/1455568357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 -- 172.21.15.50:0/1455568357 >> 172.21.15.50:0/1455568357 conn(0x7f028c071110 msgr2=0x7f028c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 -- 172.21.15.50:0/1455568357 shutdown_connections 2023-10-19T10:36:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.229+0000 7f0292841640 1 -- 172.21.15.50:0/1455568357 wait complete. 2023-10-19T10:36:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.230+0000 7f0292841640 1 Processor -- start 2023-10-19T10:36:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.230+0000 7f0292841640 1 -- start start 2023-10-19T10:36:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0292841640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 0x7f028c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0292841640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0292841640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0d04c0 0x7f028c0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0292841640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0c79e0 con 0x7f028c0b1540 2023-10-19T10:36:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0292841640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0b3040 con 0x7f028c0d04c0 2023-10-19T10:36:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0292841640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0c7c80 con 0x7f028c0cf370 2023-10-19T10:36:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f028bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 0x7f028c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f028bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 0x7f028c0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60224/0 (socket says 172.21.15.50:60224) 2023-10-19T10:36:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f028bfff640 1 -- 172.21.15.50:0/3366035128 learned_addr learned my addr 172.21.15.50:0/3366035128 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f028b7fe640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.231+0000 7f0290dde640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0d04c0 0x7f028c0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f028bfff640 1 -- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 msgr2=0x7f028c1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f028bfff640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f028bfff640 1 -- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0d04c0 msgr2=0x7f028c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f028bfff640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0d04c0 0x7f028c0b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f028bfff640 1 -- 172.21.15.50:0/3366035128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800094d0 con 0x7f028c0b1540 2023-10-19T10:36:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f0290dde640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0d04c0 0x7f028c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f028bfff640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 0x7f028c0ca6e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e810 tx=0x7f027c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c004230 con 0x7f028c0b1540 2023-10-19T10:36:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c004390 con 0x7f028c0b1540 2023-10-19T10:36:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c0c7f60 con 0x7f028c0b1540 2023-10-19T10:36:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c005bb0 con 0x7f028c0b1540 2023-10-19T10:36:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.232+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c0cac20 con 0x7f028c0b1540 2023-10-19T10:36:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.233+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c073990 con 0x7f028c0b1540 2023-10-19T10:36:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.233+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f027c005d10 con 0x7f028c0b1540 2023-10-19T10:36:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.234+0000 7f0288ff9640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f027806a050 0x7f027806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:56.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.234+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f027c088590 con 0x7f028c0b1540 2023-10-19T10:36:56.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.234+0000 7f028b7fe640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f027806a050 0x7f027806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:56.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.235+0000 7f028b7fe640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f027806a050 0x7f027806c510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0280036000 tx=0x7f0280008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:56.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.238+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f027c053ba0 con 0x7f028c0b1540 2023-10-19T10:36:56.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:56.416+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f028c06b720 con 0x7f028c0b1540 2023-10-19T10:36:58.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.326+0000 7f0288ff9640 1 -- 172.21.15.50:0/3366035128 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) v1 ==== 228+0+0 (secure 0 0 0) 0x7f027c0571c0 con 0x7f028c0b1540 2023-10-19T10:36:58.329 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' created 2023-10-19T10:36:58.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.331+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f027806a050 msgr2=0x7f027806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.331+0000 7f0292841640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f027806a050 0x7f027806c510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0280036000 tx=0x7f0280008030 comp rx=0 tx=0).stop 2023-10-19T10:36:58.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.331+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 msgr2=0x7f028c0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.331+0000 7f0292841640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 0x7f028c0ca6e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e810 tx=0x7f027c00ecd0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.333+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 shutdown_connections 2023-10-19T10:36:58.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.333+0000 7f0292841640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f027806a050 0x7f027806c510 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.333+0000 7f0292841640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c0d04c0 0x7f028c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.333+0000 7f0292841640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0cf370 0x7f028c1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.333+0000 7f0292841640 1 --2- 172.21.15.50:0/3366035128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f028c0b1540 0x7f028c0ca6e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.333+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 >> 172.21.15.50:0/3366035128 conn(0x7f028c071110 msgr2=0x7f028c160970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:58.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.334+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 shutdown_connections 2023-10-19T10:36:58.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.334+0000 7f0292841640 1 -- 172.21.15.50:0/3366035128 wait complete. 2023-10-19T10:36:58.352 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-19T10:36:58.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.465+0000 7f2e4d86a640 1 Processor -- start 2023-10-19T10:36:58.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.465+0000 7f2e4d86a640 1 -- start start 2023-10-19T10:36:58.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e4d86a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2e480b55c0 0x7f2e480cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e4d86a640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e480cfd80 con 0x7f2e480cf930 2023-10-19T10:36:58.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e4d86a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e480cda30 con 0x7f2e4815c620 2023-10-19T10:36:58.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e4d86a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e480ccc10 con 0x7f2e480b55c0 2023-10-19T10:36:58.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e477fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2e480cf930 0x7f2e480d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57044/0 (socket says 172.21.15.50:57044) 2023-10-19T10:36:58.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e477fe640 1 -- 172.21.15.50:0/3506020065 learned_addr learned my addr 172.21.15.50:0/3506020065 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:36:58.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430827749 0 0) 0x7f2e40001490 con 0x7f2e480cf930 2023-10-19T10:36:58.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e30003520 con 0x7f2e480cf930 2023-10-19T10:36:58.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e467fc640 1 --2- 172.21.15.50:0/3506020065 >> v2:172.21.15.183:3300/0 conn(0x7f2e480b55c0 0x7f2e480cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850086401 0 0) 0x7f2e30003520 con 0x7f2e480cf930 2023-10-19T10:36:58.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 >> v2:172.21.15.183:3300/0 conn(0x7f2e480b55c0 msgr2=0x7f2e480cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:58.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 --2- 172.21.15.50:0/3506020065 >> v2:172.21.15.183:3300/0 conn(0x7f2e480b55c0 0x7f2e480cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 >> v1:172.21.15.183:6789/0 conn(0x7f2e4815c620 legacy=0x7f2e480b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48161110 con 0x7f2e480cf930 2023-10-19T10:36:58.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f2e40001b80 con 0x7f2e480cf930 2023-10-19T10:36:58.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 >> v1:172.21.15.50:6789/0 conn(0x7f2e480cf930 legacy=0x7f2e480d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e300047c0 0x7f2e30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e30005180 0x7f2e3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480ccc10 con 0x7f2e30003970 2023-10-19T10:36:58.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3000c020 con 0x7f2e300047c0 2023-10-19T10:36:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3000c1e0 con 0x7f2e30005180 2023-10-19T10:36:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.466+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e467fc640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e300047c0 0x7f2e30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e477fe640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e30005180 0x7f2e3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e30005180 msgr2=0x7f2e3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:58.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e30005180 0x7f2e3000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e300047c0 msgr2=0x7f2e30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:58.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e300047c0 0x7f2e30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3506020065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30003520 con 0x7f2e30003970 2023-10-19T10:36:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e477fe640 1 -- 172.21.15.50:0/3506020065 reap_dead start 2023-10-19T10:36:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.467+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e30003d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e38001750 tx=0x7f2e300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa28e55b3dae86 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380052d0 con 0x7f2e30003970 2023-10-19T10:36:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3802e050 con 0x7f2e30003970 2023-10-19T10:36:58.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e45ffb640 1 -- 172.21.15.50:0/3506020065 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38003830 con 0x7f2e30003970 2023-10-19T10:36:58.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3506020065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 msgr2=0x7f2e30003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e30003d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e38001750 tx=0x7f2e300057f0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3506020065 shutdown_connections 2023-10-19T10:36:58.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3506020065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e30003d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3506020065 >> 172.21.15.50:0/3506020065 conn(0x7f2e48071110 msgr2=0x7f2e480714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:58.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3506020065 shutdown_connections 2023-10-19T10:36:58.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.468+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3506020065 wait complete. 2023-10-19T10:36:58.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.469+0000 7f2e4d86a640 1 Processor -- start 2023-10-19T10:36:58.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 -- start start 2023-10-19T10:36:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e480ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e480b39a0 0x7f2e480cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e480b3e20 0x7f2e480cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e48154df0 con 0x7f2e30003970 2023-10-19T10:36:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480cbe70 con 0x7f2e480b39a0 2023-10-19T10:36:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e4d86a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c9690 con 0x7f2e480b3e20 2023-10-19T10:36:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e46ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e480ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e46ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e480ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39508/0 (socket says 172.21.15.50:39508) 2023-10-19T10:36:58.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.470+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3322885382 learned_addr learned my addr 172.21.15.50:0/3322885382 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:58.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e467fc640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e480b39a0 0x7f2e480cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e477fe640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e480b3e20 0x7f2e480cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e480b3e20 msgr2=0x7f2e480cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e480b3e20 0x7f2e480cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e480b39a0 msgr2=0x7f2e480cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e480b39a0 0x7f2e480cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e46ffd640 1 -- 172.21.15.50:0/3322885382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e480c9970 con 0x7f2e30003970 2023-10-19T10:36:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e46ffd640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e480ca450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e3802bc30 tx=0x7f2e38001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:58.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380076c0 con 0x7f2e30003970 2023-10-19T10:36:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38007a30 con 0x7f2e30003970 2023-10-19T10:36:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38006be0 con 0x7f2e30003970 2023-10-19T10:36:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e480c9c50 con 0x7f2e30003970 2023-10-19T10:36:58.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.471+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e480ca990 con 0x7f2e30003970 2023-10-19T10:36:58.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.472+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e480c8fc0 con 0x7f2e30003970 2023-10-19T10:36:58.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.473+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e38006d40 con 0x7f2e30003970 2023-10-19T10:36:58.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.473+0000 7f2e2ffff640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3406a050 0x7f2e3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:58.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.473+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f2e380aa410 con 0x7f2e30003970 2023-10-19T10:36:58.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.473+0000 7f2e467fc640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3406a050 0x7f2e3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:58.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.474+0000 7f2e467fc640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3406a050 0x7f2e3406c510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e4814b160 tx=0x7f2e3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:58.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.478+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2e38075830 con 0x7f2e30003970 2023-10-19T10:36:58.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:58.655+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f2e480c97d0 con 0x7f2e30003970 2023-10-19T10:36:59.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.331+0000 7f2e2ffff640 1 -- 172.21.15.50:0/3322885382 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2e480c97d0 con 0x7f2e30003970 2023-10-19T10:36:59.337 INFO:teuthology.orchestra.run.smithi050.stderr:set pool 5 allow_ec_overwrites to true 2023-10-19T10:36:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.343+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3406a050 msgr2=0x7f2e3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:59.348 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.343+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3406a050 0x7f2e3406c510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e4814b160 tx=0x7f2e3c008040 comp rx=0 tx=0).stop 2023-10-19T10:36:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.343+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 msgr2=0x7f2e480ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.343+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e480ca450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e3802bc30 tx=0x7f2e38001120 comp rx=0 tx=0).stop 2023-10-19T10:36:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.344+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 shutdown_connections 2023-10-19T10:36:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.344+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e3406a050 0x7f2e3406c510 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.344+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e480b3e20 0x7f2e480cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.344+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e480b39a0 0x7f2e480cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.344+0000 7f2e4d86a640 1 --2- 172.21.15.50:0/3322885382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e30003970 0x7f2e480ca450 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e3802bc30 tx=0x7f2e38001120 comp rx=0 tx=0).stop 2023-10-19T10:36:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.344+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 >> 172.21.15.50:0/3322885382 conn(0x7f2e48071110 msgr2=0x7f2e48160900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.347+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 shutdown_connections 2023-10-19T10:36:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.350+0000 7f2e4d86a640 1 -- 172.21.15.50:0/3322885382 wait complete. 2023-10-19T10:36:59.373 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-19T10:36:59.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.483+0000 7f1008363640 1 Processor -- start 2023-10-19T10:36:59.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.483+0000 7f1008363640 1 -- start start 2023-10-19T10:36:59.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f1008363640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f100015c620 0x7f10000b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:59.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f1008363640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10000b82f0 0x7f10000cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:59.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f1008363640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10000cda30 con 0x7f10000b55c0 2023-10-19T10:36:59.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f1008363640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10000ccc10 con 0x7f100015c620 2023-10-19T10:36:59.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f1008363640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1000161110 con 0x7f10000b82f0 2023-10-19T10:36:59.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f100015c620 0x7f10000b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:59.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f100015c620 0x7f10000b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39522/0 (socket says 172.21.15.50:39522) 2023-10-19T10:36:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f1006900640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f10000b55c0 0x7f10000b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57048/0 (socket says 172.21.15.50:57048) 2023-10-19T10:36:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 -- 172.21.15.50:0/1202788184 learned_addr learned my addr 172.21.15.50:0/1202788184 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10058fe640 1 --2- 172.21.15.50:0/1202788184 >> v2:172.21.15.183:3300/0 conn(0x7f10000b82f0 0x7f10000cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 -- 172.21.15.50:0/1202788184 >> v2:172.21.15.183:3300/0 conn(0x7f10000b82f0 msgr2=0x7f10000cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:36:59.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 --2- 172.21.15.50:0/1202788184 >> v2:172.21.15.183:3300/0 conn(0x7f10000b82f0 0x7f10000cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 -- 172.21.15.50:0/1202788184 >> v1:172.21.15.50:6789/0 conn(0x7f10000b55c0 legacy=0x7f10000b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:59.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10060ff640 1 -- 172.21.15.50:0/1202788184 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000cce20 con 0x7f100015c620 2023-10-19T10:36:59.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.484+0000 7f10050fd640 1 -- 172.21.15.50:0/1202788184 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557175001 0 0) 0x7f10000cda30 con 0x7f10000b55c0 2023-10-19T10:36:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.485+0000 7f10060ff640 1 --2- 172.21.15.50:0/1202788184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10000b7db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10000b50e0 tx=0x7f0ff802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4e6fd025e7dac3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.485+0000 7f10050fd640 1 -- 172.21.15.50:0/1202788184 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8030860 con 0x7f100015c620 2023-10-19T10:36:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.485+0000 7f10050fd640 1 -- 172.21.15.50:0/1202788184 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff80309c0 con 0x7f100015c620 2023-10-19T10:36:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.485+0000 7f10050fd640 1 -- 172.21.15.50:0/1202788184 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8030d50 con 0x7f100015c620 2023-10-19T10:36:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 -- 172.21.15.50:0/1202788184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 msgr2=0x7f10000b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 --2- 172.21.15.50:0/1202788184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10000b7db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10000b50e0 tx=0x7f0ff802b9a0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 -- 172.21.15.50:0/1202788184 shutdown_connections 2023-10-19T10:36:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 --2- 172.21.15.50:0/1202788184 >> v2:172.21.15.183:3300/0 conn(0x7f10000b82f0 0x7f10000cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 --2- 172.21.15.50:0/1202788184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10000b7db0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 -- 172.21.15.50:0/1202788184 >> 172.21.15.50:0/1202788184 conn(0x7f1000071110 msgr2=0x7f10000714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:36:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 -- 172.21.15.50:0/1202788184 shutdown_connections 2023-10-19T10:36:59.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.486+0000 7f1008363640 1 -- 172.21.15.50:0/1202788184 wait complete. 2023-10-19T10:36:59.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.487+0000 7f1008363640 1 Processor -- start 2023-10-19T10:36:59.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.487+0000 7f1008363640 1 -- start start 2023-10-19T10:36:59.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1008363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10000b55c0 0x7f100015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1008363640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10000b82f0 0x7f10000ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1008363640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10001505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1008363640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10000ca5c0 con 0x7f100015c620 2023-10-19T10:36:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1008363640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100015e290 con 0x7f10000b55c0 2023-10-19T10:36:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1008363640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1000150b30 con 0x7f10000b82f0 2023-10-19T10:36:59.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10001505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:59.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10001505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39530/0 (socket says 172.21.15.50:39530) 2023-10-19T10:36:59.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 -- 172.21.15.50:0/1701855603 learned_addr learned my addr 172.21.15.50:0/1701855603 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:36:59.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f10060ff640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10000b55c0 0x7f100015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:59.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f10058fe640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10000b82f0 0x7f10000ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:59.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 -- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10000b82f0 msgr2=0x7f10000ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10000b82f0 0x7f10000ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 -- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10000b55c0 msgr2=0x7f100015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:36:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10000b55c0 0x7f100015d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:36:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 -- 172.21.15.50:0/1701855603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80094d0 con 0x7f100015c620 2023-10-19T10:36:59.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f10060ff640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10000b55c0 0x7f100015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:36:59.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.488+0000 7f1006900640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10001505f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c700 tx=0x7f0ff000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:59.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.489+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000ce10 con 0x7f100015c620 2023-10-19T10:36:59.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.489+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000a640 con 0x7f100015c620 2023-10-19T10:36:59.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.489+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10000ca7d0 con 0x7f100015c620 2023-10-19T10:36:59.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.489+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000a9f0 con 0x7f100015c620 2023-10-19T10:36:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.489+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10000cad90 con 0x7f100015c620 2023-10-19T10:36:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.490+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100015e490 con 0x7f100015c620 2023-10-19T10:36:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.490+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ff000ab50 con 0x7f100015c620 2023-10-19T10:36:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.490+0000 7f0ffeffd640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fec06a050 0x7f0fec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:36:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.491+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f0ff0087c00 con 0x7f100015c620 2023-10-19T10:36:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.491+0000 7f10060ff640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fec06a050 0x7f0fec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:36:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.491+0000 7f10060ff640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fec06a050 0x7f0fec06c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8036000 tx=0x7f0ff8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:36:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.495+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f100015e490 con 0x7f100015c620 2023-10-19T10:36:59.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:36:59.712+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f100015e430 con 0x7f100015c620 2023-10-19T10:37:00.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.332+0000 7f0ffeffd640 1 -- 172.21.15.50:0/1701855603 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0ff00180b0 con 0x7f100015c620 2023-10-19T10:37:00.335 INFO:teuthology.orchestra.run.smithi050.stderr:added data pool 5 to fsmap 2023-10-19T10:37:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fec06a050 msgr2=0x7f0fec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fec06a050 0x7f0fec06c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8036000 tx=0x7f0ff8008030 comp rx=0 tx=0).stop 2023-10-19T10:37:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 msgr2=0x7f10001505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10001505f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c700 tx=0x7f0ff000cbc0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 shutdown_connections 2023-10-19T10:37:00.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fec06a050 0x7f0fec06c510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f100015c620 0x7f10001505f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10000b82f0 0x7f10000ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 --2- 172.21.15.50:0/1701855603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10000b55c0 0x7f100015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 >> 172.21.15.50:0/1701855603 conn(0x7f1000071110 msgr2=0x7f100015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:00.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 shutdown_connections 2023-10-19T10:37:00.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.335+0000 7f1008363640 1 -- 172.21.15.50:0/1701855603 wait complete. 2023-10-19T10:37:00.350 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:37:00.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.460+0000 7fe876df2640 1 Processor -- start 2023-10-19T10:37:00.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.460+0000 7fe876df2640 1 -- start start 2023-10-19T10:37:00.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe876df2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe876df2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe87015c950 0x7fe8700cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe876df2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8700cde60 con 0x7fe87006b760 2023-10-19T10:37:00.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe876df2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87006aa30 con 0x7fe87015c950 2023-10-19T10:37:00.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe876df2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87006b2d0 con 0x7fe87015c570 2023-10-19T10:37:00.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe86ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe87015c950 0x7fe8700cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe87538f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe87006b760 0x7fe8700d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57052/0 (socket says 172.21.15.50:57052) 2023-10-19T10:37:00.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe87538f640 1 -- 172.21.15.50:0/3305108559 learned_addr learned my addr 172.21.15.50:0/3305108559 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:00.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe86ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe87015c950 0x7fe8700cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39542/0 (socket says 172.21.15.50:39542) 2023-10-19T10:37:00.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.461+0000 7fe874b8e640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86ffff640 1 -- 172.21.15.50:0/3305108559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 msgr2=0x7fe8700d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:00.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86ffff640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86ffff640 1 -- 172.21.15.50:0/3305108559 >> v1:172.21.15.50:6789/0 conn(0x7fe87006b760 legacy=0x7fe8700d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86ffff640 1 -- 172.21.15.50:0/3305108559 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700ce090 con 0x7fe87015c950 2023-10-19T10:37:00.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86f7fe640 1 -- 172.21.15.50:0/3305108559 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961372531 0 0) 0x7fe8700cde60 con 0x7fe87006b760 2023-10-19T10:37:00.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe874b8e640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:00.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86ffff640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8700cd920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8640097a0 tx=0x7fe86402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5189071ab03ff764 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:00.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86f7fe640 1 -- 172.21.15.50:0/3305108559 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86402e070 con 0x7fe87015c950 2023-10-19T10:37:00.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86f7fe640 1 -- 172.21.15.50:0/3305108559 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8640315a0 con 0x7fe87015c950 2023-10-19T10:37:00.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.462+0000 7fe86f7fe640 1 -- 172.21.15.50:0/3305108559 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864031930 con 0x7fe87015c950 2023-10-19T10:37:00.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 -- 172.21.15.50:0/3305108559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 msgr2=0x7fe8700cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8700cd920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8640097a0 tx=0x7fe86402b840 comp rx=0 tx=0).stop 2023-10-19T10:37:00.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 -- 172.21.15.50:0/3305108559 shutdown_connections 2023-10-19T10:37:00.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8700cd920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 --2- 172.21.15.50:0/3305108559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 -- 172.21.15.50:0/3305108559 >> 172.21.15.50:0/3305108559 conn(0x7fe870089ff0 msgr2=0x7fe87009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:00.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 -- 172.21.15.50:0/3305108559 shutdown_connections 2023-10-19T10:37:00.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.463+0000 7fe876df2640 1 -- 172.21.15.50:0/3305108559 wait complete. 2023-10-19T10:37:00.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.464+0000 7fe876df2640 1 Processor -- start 2023-10-19T10:37:00.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.464+0000 7fe876df2640 1 -- start start 2023-10-19T10:37:00.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe876df2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe87006b760 0x7fe8700cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe876df2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe876df2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8701597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe876df2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700cc410 con 0x7fe87015c950 2023-10-19T10:37:00.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe876df2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700b79e0 con 0x7fe87006b760 2023-10-19T10:37:00.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe876df2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700b7cc0 con 0x7fe87015c570 2023-10-19T10:37:00.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe87538f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8701597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe87538f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8701597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39544/0 (socket says 172.21.15.50:39544) 2023-10-19T10:37:00.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe87538f640 1 -- 172.21.15.50:0/2745938287 learned_addr learned my addr 172.21.15.50:0/2745938287 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:00.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe86ffff640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe874b8e640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe87006b760 0x7fe8700cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe87538f640 1 -- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 msgr2=0x7fe8700cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.465+0000 7fe87538f640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe87538f640 1 -- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe87006b760 msgr2=0x7fe8700cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe87538f640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe87006b760 0x7fe8700cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe87538f640 1 -- 172.21.15.50:0/2745938287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe864009400 con 0x7fe87015c950 2023-10-19T10:37:00.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe86ffff640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:00.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe874b8e640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe87006b760 0x7fe8700cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:00.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe87538f640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8701597a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c780 tx=0x7fe86800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:00.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800ce30 con 0x7fe87015c950 2023-10-19T10:37:00.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86800a640 con 0x7fe87015c950 2023-10-19T10:37:00.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8700b7fa0 con 0x7fe87015c950 2023-10-19T10:37:00.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800a9f0 con 0x7fe87015c950 2023-10-19T10:37:00.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.466+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8700b5580 con 0x7fe87015c950 2023-10-19T10:37:00.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.467+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe870073990 con 0x7fe87015c950 2023-10-19T10:37:00.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.467+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe86800ab50 con 0x7fe87015c950 2023-10-19T10:37:00.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.468+0000 7fe86d7fa640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe85c06a050 0x7fe85c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.468+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe868087e40 con 0x7fe87015c950 2023-10-19T10:37:00.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.468+0000 7fe874b8e640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe85c06a050 0x7fe85c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.469+0000 7fe874b8e640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe85c06a050 0x7fe85c06c510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe860009950 tx=0x7fe860008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:00.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.472+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe868053110 con 0x7fe87015c950 2023-10-19T10:37:00.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.682+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8700d3480 con 0x7fe87015c950 2023-10-19T10:37:00.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.683+0000 7fe86d7fa640 1 -- 172.21.15.50:0/2745938287 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4877 (secure 0 0 0) 0x7fe868018030 con 0x7fe87015c950 2023-10-19T10:37:00.686 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:00.686 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:00.329659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-19T10:37:00.686 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 7 2023-10-19T10:37:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe85c06a050 msgr2=0x7fe85c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe85c06a050 0x7fe85c06c510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe860009950 tx=0x7fe860008040 comp rx=0 tx=0).stop 2023-10-19T10:37:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 msgr2=0x7fe8701597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8701597a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c780 tx=0x7fe86800cc40 comp rx=0 tx=0).stop 2023-10-19T10:37:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 shutdown_connections 2023-10-19T10:37:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe85c06a050 0x7fe85c06c510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87015c950 0x7fe8701597a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe87015c570 0x7fe8700cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 --2- 172.21.15.50:0/2745938287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe87006b760 0x7fe8700cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 >> 172.21.15.50:0/2745938287 conn(0x7fe870089ff0 msgr2=0x7fe8700d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:00.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 shutdown_connections 2023-10-19T10:37:00.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.685+0000 7fe876df2640 1 -- 172.21.15.50:0/2745938287 wait complete. 2023-10-19T10:37:00.698 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:00.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.801+0000 7f8998028640 1 Processor -- start 2023-10-19T10:37:00.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 -- start start 2023-10-19T10:37:00.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8990156c50 0x7f89900cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89900ce360 0x7f8990160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f899015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89901610a0 con 0x7f89900ce360 2023-10-19T10:37:00.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8990157030 con 0x7f8990156c50 2023-10-19T10:37:00.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f8998028640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899015cac0 con 0x7f8990160890 2023-10-19T10:37:00.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.802+0000 7f89955c3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89900ce360 0x7f8990160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89900ce360 0x7f8990160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39550/0 (socket says 172.21.15.50:39550) 2023-10-19T10:37:00.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 -- 172.21.15.50:0/3589945622 learned_addr learned my addr 172.21.15.50:0/3589945622 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:00.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f8995dc4640 1 --2- 172.21.15.50:0/3589945622 >> v2:172.21.15.183:3300/0 conn(0x7f8990156c50 0x7f89900cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 -- 172.21.15.50:0/3589945622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 msgr2=0x7f899015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:00.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 --2- 172.21.15.50:0/3589945622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f899015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 -- 172.21.15.50:0/3589945622 >> v2:172.21.15.183:3300/0 conn(0x7f8990156c50 msgr2=0x7f89900cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:00.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 --2- 172.21.15.50:0/3589945622 >> v2:172.21.15.183:3300/0 conn(0x7f8990156c50 0x7f89900cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 -- 172.21.15.50:0/3589945622 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899015da90 con 0x7f89900ce360 2023-10-19T10:37:00.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.803+0000 7f89955c3640 1 --2- 172.21.15.50:0/3589945622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89900ce360 0x7f8990160350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8990070270 tx=0x7f898002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8223fc22086e98dd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:00.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8994dc2640 1 -- 172.21.15.50:0/3589945622 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89800317e0 con 0x7f89900ce360 2023-10-19T10:37:00.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8994dc2640 1 -- 172.21.15.50:0/3589945622 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8980031940 con 0x7f89900ce360 2023-10-19T10:37:00.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8994dc2640 1 -- 172.21.15.50:0/3589945622 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980031cd0 con 0x7f89900ce360 2023-10-19T10:37:00.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 -- 172.21.15.50:0/3589945622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89900ce360 msgr2=0x7f8990160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 --2- 172.21.15.50:0/3589945622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89900ce360 0x7f8990160350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8990070270 tx=0x7f898002b880 comp rx=0 tx=0).stop 2023-10-19T10:37:00.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 -- 172.21.15.50:0/3589945622 shutdown_connections 2023-10-19T10:37:00.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 --2- 172.21.15.50:0/3589945622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f899015f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 --2- 172.21.15.50:0/3589945622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89900ce360 0x7f8990160350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 --2- 172.21.15.50:0/3589945622 >> v2:172.21.15.183:3300/0 conn(0x7f8990156c50 0x7f89900cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 -- 172.21.15.50:0/3589945622 >> 172.21.15.50:0/3589945622 conn(0x7f899006f3f0 msgr2=0x7f899006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:00.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 -- 172.21.15.50:0/3589945622 shutdown_connections 2023-10-19T10:37:00.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.804+0000 7f8998028640 1 -- 172.21.15.50:0/3589945622 wait complete. 2023-10-19T10:37:00.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.805+0000 7f8998028640 1 Processor -- start 2023-10-19T10:37:00.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.805+0000 7f8998028640 1 -- start start 2023-10-19T10:37:00.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8998028640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89900ce360 0x7f89901f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8998028640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 0x7f89900b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8998028640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f89900b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8998028640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8990150a80 con 0x7f8990156c50 2023-10-19T10:37:00.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8998028640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8990144240 con 0x7f89900ce360 2023-10-19T10:37:00.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8998028640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8990144520 con 0x7f8990160890 2023-10-19T10:37:00.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f89955c3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 0x7f89900b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f89955c3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 0x7f89900b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39562/0 (socket says 172.21.15.50:39562) 2023-10-19T10:37:00.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f89955c3640 1 -- 172.21.15.50:0/85639792 learned_addr learned my addr 172.21.15.50:0/85639792 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:00.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f8995dc4640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89900ce360 0x7f89901f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f89955c3640 1 -- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 msgr2=0x7f89900b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.806+0000 7f89965c5640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f89900b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89955c3640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f89900b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89955c3640 1 -- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89900ce360 msgr2=0x7f89901f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89955c3640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89900ce360 0x7f89901f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89955c3640 1 -- 172.21.15.50:0/85639792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800094d0 con 0x7f8990156c50 2023-10-19T10:37:00.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f8995dc4640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89900ce360 0x7f89901f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:00.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89955c3640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 0x7f89900b48f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8980000c00 tx=0x7f89800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:00.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89800329c0 con 0x7f8990156c50 2023-10-19T10:37:00.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8980032b40 con 0x7f8990156c50 2023-10-19T10:37:00.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8990144800 con 0x7f8990156c50 2023-10-19T10:37:00.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89800453f0 con 0x7f8990156c50 2023-10-19T10:37:00.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.807+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89901f5ac0 con 0x7f8990156c50 2023-10-19T10:37:00.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.808+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8990150bf0 con 0x7f8990156c50 2023-10-19T10:37:00.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.808+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f898004d3f0 con 0x7f8990156c50 2023-10-19T10:37:00.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.809+0000 7f89867fc640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f897406a050 0x7f897406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:00.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.809+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f89800aa840 con 0x7f8990156c50 2023-10-19T10:37:00.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.809+0000 7f8995dc4640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f897406a050 0x7f897406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:00.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.810+0000 7f8995dc4640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f897406a050 0x7f897406c510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f898c006fd0 tx=0x7f898c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:00.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.813+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f8980075ce0 con 0x7f8990156c50 2023-10-19T10:37:00.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.992+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8990144660 con 0x7f8990156c50 2023-10-19T10:37:00.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.993+0000 7f89867fc640 1 -- 172.21.15.50:0/85639792 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f8990144660 con 0x7f8990156c50 2023-10-19T10:37:00.996 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:00.996 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:00.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f897406a050 msgr2=0x7f897406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f897406a050 0x7f897406c510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f898c006fd0 tx=0x7f898c008040 comp rx=0 tx=0).stop 2023-10-19T10:37:00.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 msgr2=0x7f89900b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:00.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 0x7f89900b48f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8980000c00 tx=0x7f89800046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 shutdown_connections 2023-10-19T10:37:00.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f897406a050 0x7f897406c510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:00.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8990160890 0x7f89900b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990156c50 0x7f89900b48f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 --2- 172.21.15.50:0/85639792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89900ce360 0x7f89901f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 >> 172.21.15.50:0/85639792 conn(0x7f899006f3f0 msgr2=0x7f899006a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 shutdown_connections 2023-10-19T10:37:01.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:00.995+0000 7f8998028640 1 -- 172.21.15.50:0/85639792 wait complete. 2023-10-19T10:37:01.009 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:01.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.113+0000 7fc955303640 1 Processor -- start 2023-10-19T10:37:01.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.113+0000 7fc955303640 1 -- start start 2023-10-19T10:37:01.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc955303640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc9500d0e30 0x7fc95015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc955303640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95006aa30 con 0x7fc9500cd920 2023-10-19T10:37:01.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc955303640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95006b2d0 con 0x7fc95015c570 2023-10-19T10:37:01.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc955303640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95015ce90 con 0x7fc9500d0e30 2023-10-19T10:37:01.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc9500cd920 0x7fc9500d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57062/0 (socket says 172.21.15.50:57062) 2023-10-19T10:37:01.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94f7fe640 1 -- 172.21.15.50:0/1236306090 learned_addr learned my addr 172.21.15.50:0/1236306090 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:01.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc9467fc640 1 --2- 172.21.15.50:0/1236306090 >> v2:172.21.15.183:3300/0 conn(0x7fc9500d0e30 0x7fc95015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969078835 0 0) 0x7fc95006aa30 con 0x7fc9500cd920 2023-10-19T10:37:01.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc93c003520 con 0x7fc9500cd920 2023-10-19T10:37:01.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730650981 0 0) 0x7fc93c003520 con 0x7fc9500cd920 2023-10-19T10:37:01.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 >> v2:172.21.15.183:3300/0 conn(0x7fc9500d0e30 msgr2=0x7fc95015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 --2- 172.21.15.50:0/1236306090 >> v2:172.21.15.183:3300/0 conn(0x7fc9500d0e30 0x7fc95015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 >> v1:172.21.15.183:6789/0 conn(0x7fc95015c570 legacy=0x7fc95006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500d1270 con 0x7fc9500cd920 2023-10-19T10:37:01.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fc9480019f0 con 0x7fc9500cd920 2023-10-19T10:37:01.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 >> v1:172.21.15.50:6789/0 conn(0x7fc9500cd920 legacy=0x7fc9500d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc93c003970 0x7fc93c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c0047c0 0x7fc93c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c005180 0x7fc93c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95015ce90 con 0x7fc93c0047c0 2023-10-19T10:37:01.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00c120 con 0x7fc93c005180 2023-10-19T10:37:01.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.114+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00c2e0 con 0x7fc93c003970 2023-10-19T10:37:01.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc9467fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c0047c0 0x7fc93c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc94f7fe640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c005180 0x7fc93c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc94effd640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc93c003970 0x7fc93c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc9467fc640 1 -- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc93c003970 msgr2=0x7fc93c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc9467fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc93c003970 0x7fc93c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc9467fc640 1 -- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c005180 msgr2=0x7fc93c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc9467fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c005180 0x7fc93c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc94f7fe640 1 -- 172.21.15.50:0/1236306090 reap_dead start 2023-10-19T10:37:01.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.115+0000 7fc9467fc640 1 -- 172.21.15.50:0/1236306090 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c003520 con 0x7fc93c0047c0 2023-10-19T10:37:01.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc9467fc640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c0047c0 0x7fc93c004c40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc938009ac0 tx=0x7fc93c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1998585e5b7f4b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93802c6b0 con 0x7fc93c0047c0 2023-10-19T10:37:01.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93802c810 con 0x7fc93c0047c0 2023-10-19T10:37:01.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc94e7fc640 1 -- 172.21.15.50:0/1236306090 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93802cba0 con 0x7fc93c0047c0 2023-10-19T10:37:01.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 -- 172.21.15.50:0/1236306090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c0047c0 msgr2=0x7fc93c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c0047c0 0x7fc93c004c40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc938009ac0 tx=0x7fc93c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 -- 172.21.15.50:0/1236306090 shutdown_connections 2023-10-19T10:37:01.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 --2- 172.21.15.50:0/1236306090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c0047c0 0x7fc93c004c40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 -- 172.21.15.50:0/1236306090 >> 172.21.15.50:0/1236306090 conn(0x7fc950089ff0 msgr2=0x7fc95009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 -- 172.21.15.50:0/1236306090 shutdown_connections 2023-10-19T10:37:01.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.116+0000 7fc955303640 1 -- 172.21.15.50:0/1236306090 wait complete. 2023-10-19T10:37:01.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.117+0000 7fc955303640 1 Processor -- start 2023-10-19T10:37:01.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 -- start start 2023-10-19T10:37:01.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0047c0 0x7fc95015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 0x7fc9501f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc95015f210 0x7fc9501f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501f2b70 con 0x7fc95015fb40 2023-10-19T10:37:01.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95015cfd0 con 0x7fc93c0047c0 2023-10-19T10:37:01.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc955303640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95015d220 con 0x7fc95015f210 2023-10-19T10:37:01.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc9467fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 0x7fc9501f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc9467fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 0x7fc9501f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39580/0 (socket says 172.21.15.50:39580) 2023-10-19T10:37:01.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc9467fc640 1 -- 172.21.15.50:0/331728682 learned_addr learned my addr 172.21.15.50:0/331728682 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:01.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc94effd640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0047c0 0x7fc95015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.118+0000 7fc94f7fe640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc95015f210 0x7fc9501f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc9467fc640 1 -- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc95015f210 msgr2=0x7fc9501f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc9467fc640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc95015f210 0x7fc9501f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc9467fc640 1 -- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0047c0 msgr2=0x7fc95015e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc9467fc640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0047c0 0x7fc95015e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc9467fc640 1 -- 172.21.15.50:0/331728682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95015d500 con 0x7fc95015fb40 2023-10-19T10:37:01.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc9467fc640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 0x7fc9501f2630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc938037040 tx=0x7fc938004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938003810 con 0x7fc95015fb40 2023-10-19T10:37:01.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938003970 con 0x7fc95015fb40 2023-10-19T10:37:01.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938003d20 con 0x7fc95015fb40 2023-10-19T10:37:01.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95015d7e0 con 0x7fc95015fb40 2023-10-19T10:37:01.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.119+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95015dda0 con 0x7fc95015fb40 2023-10-19T10:37:01.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.120+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95015df30 con 0x7fc95015fb40 2023-10-19T10:37:01.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.120+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc938034440 con 0x7fc95015fb40 2023-10-19T10:37:01.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.121+0000 7fc947fff640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc93006a050 0x7fc93006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.121+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc9380aa8c0 con 0x7fc95015fb40 2023-10-19T10:37:01.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.121+0000 7fc94effd640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc93006a050 0x7fc93006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.122+0000 7fc94effd640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc93006a050 0x7fc93006c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc940000c00 tx=0x7fc940009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.126+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc938075df0 con 0x7fc95015fb40 2023-10-19T10:37:01.287 INFO:tasks.ceph.mds.b.smithi183.stderr:2023-10-19T10:37:01.284+0000 7f7bead18640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-19T10:37:01.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.300+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc950073990 con 0x7fc95015fb40 2023-10-19T10:37:01.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.301+0000 7fc947fff640 1 -- 172.21.15.50:0/331728682 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7fc938079410 con 0x7fc95015fb40 2023-10-19T10:37:01.304 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:01.305 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:01.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc93006a050 msgr2=0x7fc93006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc93006a050 0x7fc93006c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc940000c00 tx=0x7fc940009040 comp rx=0 tx=0).stop 2023-10-19T10:37:01.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 msgr2=0x7fc9501f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 0x7fc9501f2630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc938037040 tx=0x7fc938004730 comp rx=0 tx=0).stop 2023-10-19T10:37:01.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 shutdown_connections 2023-10-19T10:37:01.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc93006a050 0x7fc93006c510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc95015f210 0x7fc9501f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95015fb40 0x7fc9501f2630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 --2- 172.21.15.50:0/331728682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0047c0 0x7fc95015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 >> 172.21.15.50:0/331728682 conn(0x7fc950089ff0 msgr2=0x7fc9500d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.303+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 shutdown_connections 2023-10-19T10:37:01.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.304+0000 7fc955303640 1 -- 172.21.15.50:0/331728682 wait complete. 2023-10-19T10:37:01.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:37:01.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.441+0000 7f3444fa0640 1 Processor -- start 2023-10-19T10:37:01.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f3444fa0640 1 -- start start 2023-10-19T10:37:01.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f3444fa0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f34401532e0 0x7f34400657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f3444fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 0x7f3440159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f3444fa0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3440153b40 con 0x7f34400ca3a0 2023-10-19T10:37:01.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f3444fa0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344006ce30 con 0x7f34401532e0 2023-10-19T10:37:01.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f3444fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3440159b80 con 0x7f34401536c0 2023-10-19T10:37:01.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f343ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f34400ca3a0 0x7f34400ca7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57074/0 (socket says 172.21.15.50:57074) 2023-10-19T10:37:01.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.442+0000 7f343ffff640 1 -- 172.21.15.50:0/2482940617 learned_addr learned my addr 172.21.15.50:0/2482940617 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:01.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 --2- 172.21.15.50:0/2482940617 >> v2:172.21.15.183:3300/0 conn(0x7f34401532e0 0x7f34400657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573847746 0 0) 0x7f3440153b40 con 0x7f34400ca3a0 2023-10-19T10:37:01.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3428005730 con 0x7f34400ca3a0 2023-10-19T10:37:01.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343effd640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 0x7f3440159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145138361 0 0) 0x7f3428005730 con 0x7f34400ca3a0 2023-10-19T10:37:01.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 msgr2=0x7f3440159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 0x7f3440159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 >> v2:172.21.15.183:3300/0 conn(0x7f34401532e0 msgr2=0x7f34400657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 --2- 172.21.15.50:0/2482940617 >> v2:172.21.15.183:3300/0 conn(0x7f34401532e0 0x7f34400657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3440159e30 con 0x7f34400ca3a0 2023-10-19T10:37:01.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f34380019f0 con 0x7f34400ca3a0 2023-10-19T10:37:01.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 >> v1:172.21.15.50:6789/0 conn(0x7f34400ca3a0 legacy=0x7f34400ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f3428005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 0x7f34280069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3428006f10 0x7f3428007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344006ce30 con 0x7f3428005b80 2023-10-19T10:37:01.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3440159b80 con 0x7f3428006f10 2023-10-19T10:37:01.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342800db90 con 0x7f34401536c0 2023-10-19T10:37:01.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f3428005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343effd640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 0x7f34280069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343ffff640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3428006f10 0x7f3428007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 -- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 msgr2=0x7f34280069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34401536c0 0x7f34280069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 -- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3428006f10 msgr2=0x7f3428007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3428006f10 0x7f3428007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.443+0000 7f343f7fe640 1 -- 172.21.15.50:0/2482940617 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428005730 con 0x7f3428005b80 2023-10-19T10:37:01.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.444+0000 7f343f7fe640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f3428005fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3430009af0 tx=0x7f3428007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1028d1858f043351 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.444+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343002c710 con 0x7f3428005b80 2023-10-19T10:37:01.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.444+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343002c870 con 0x7f3428005b80 2023-10-19T10:37:01.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.444+0000 7f343e7fc640 1 -- 172.21.15.50:0/2482940617 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343002cc00 con 0x7f3428005b80 2023-10-19T10:37:01.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f3444fa0640 1 -- 172.21.15.50:0/2482940617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 msgr2=0x7f3428005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f3444fa0640 1 --2- 172.21.15.50:0/2482940617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f3428005fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3430009af0 tx=0x7f3428007a00 comp rx=0 tx=0).stop 2023-10-19T10:37:01.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f343ffff640 1 -- 172.21.15.50:0/2482940617 reap_dead start 2023-10-19T10:37:01.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f3444fa0640 1 -- 172.21.15.50:0/2482940617 shutdown_connections 2023-10-19T10:37:01.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f3444fa0640 1 -- 172.21.15.50:0/2482940617 >> 172.21.15.50:0/2482940617 conn(0x7f34400b01d0 msgr2=0x7f34400b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f3444fa0640 1 -- 172.21.15.50:0/2482940617 shutdown_connections 2023-10-19T10:37:01.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.445+0000 7f3444fa0640 1 -- 172.21.15.50:0/2482940617 wait complete. 2023-10-19T10:37:01.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.446+0000 7f3444fa0640 1 Processor -- start 2023-10-19T10:37:01.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.446+0000 7f3444fa0640 1 -- start start 2023-10-19T10:37:01.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f3444fa0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f34400b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f3444fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f344014de70 0x7f3440149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f3444fa0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f344015a630 0x7f344015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f3444fa0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344015b000 con 0x7f3428005b80 2023-10-19T10:37:01.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f3444fa0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344014e5e0 con 0x7f344015a630 2023-10-19T10:37:01.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f3444fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344014a1a0 con 0x7f344014de70 2023-10-19T10:37:01.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f34400b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f34400b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39594/0 (socket says 172.21.15.50:39594) 2023-10-19T10:37:01.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 -- 172.21.15.50:0/3505989603 learned_addr learned my addr 172.21.15.50:0/3505989603 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:01.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343ffff640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f344015a630 0x7f344015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343effd640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f344014de70 0x7f3440149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 -- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f344014de70 msgr2=0x7f3440149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f344014de70 0x7f3440149c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 -- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f344015a630 msgr2=0x7f344015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f344015a630 0x7f344015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.447+0000 7f343f7fe640 1 -- 172.21.15.50:0/3505989603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344014e940 con 0x7f3428005b80 2023-10-19T10:37:01.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.448+0000 7f343f7fe640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f34400b23e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f343002c2b0 tx=0x7f3430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.448+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430003810 con 0x7f3428005b80 2023-10-19T10:37:01.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.448+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343002de90 con 0x7f3428005b80 2023-10-19T10:37:01.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.448+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344014ec20 con 0x7f3428005b80 2023-10-19T10:37:01.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.448+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430003b70 con 0x7f3428005b80 2023-10-19T10:37:01.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.448+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344014f1e0 con 0x7f3428005b80 2023-10-19T10:37:01.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.449+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344014f370 con 0x7f3428005b80 2023-10-19T10:37:01.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.449+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3430003cd0 con 0x7f3428005b80 2023-10-19T10:37:01.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.449+0000 7f3427fff640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f342c06a050 0x7f342c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.450+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f34300aa580 con 0x7f3428005b80 2023-10-19T10:37:01.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.450+0000 7f343effd640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f342c06a050 0x7f342c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.450+0000 7f343effd640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f342c06a050 0x7f342c06c510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3434009bc0 tx=0x7f3434009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.454+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3430075a30 con 0x7f3428005b80 2023-10-19T10:37:01.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.666+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3440066e80 con 0x7f3428005b80 2023-10-19T10:37:01.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.666+0000 7f3427fff640 1 -- 172.21.15.50:0/3505989603 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4877 (secure 0 0 0) 0x7f3430079050 con 0x7f3428005b80 2023-10-19T10:37:01.669 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:01.670 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:00.329659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-19T10:37:01.671 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 7 2023-10-19T10:37:01.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f342c06a050 msgr2=0x7f342c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f342c06a050 0x7f342c06c510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3434009bc0 tx=0x7f3434009190 comp rx=0 tx=0).stop 2023-10-19T10:37:01.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 msgr2=0x7f34400b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f34400b23e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f343002c2b0 tx=0x7f3430004730 comp rx=0 tx=0).stop 2023-10-19T10:37:01.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 shutdown_connections 2023-10-19T10:37:01.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f342c06a050 0x7f342c06c510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f344015a630 0x7f344015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f344014de70 0x7f3440149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 --2- 172.21.15.50:0/3505989603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3428005b80 0x7f34400b23e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 >> 172.21.15.50:0/3505989603 conn(0x7f34400b01d0 msgr2=0x7f34400c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 shutdown_connections 2023-10-19T10:37:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.668+0000 7f3444fa0640 1 -- 172.21.15.50:0/3505989603 wait complete. 2023-10-19T10:37:01.683 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:01.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.786+0000 7f5db5cb1640 1 Processor -- start 2023-10-19T10:37:01.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.786+0000 7f5db5cb1640 1 -- start start 2023-10-19T10:37:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5db5cb1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 0x7f5db00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5db5cb1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db00d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5db5cb1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db006aa30 con 0x7f5db015c950 2023-10-19T10:37:01.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5db5cb1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db006b2d0 con 0x7f5db015c570 2023-10-19T10:37:01.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5db5cb1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db00ce090 con 0x7f5db00cd920 2023-10-19T10:37:01.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5daf7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 0x7f5db00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5daffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db00d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.787+0000 7f5daeffd640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f5db015c950 0x7f5db006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:38268/0 (socket says 172.21.15.50:38268) 2023-10-19T10:37:01.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daf7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 0x7f5db00d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:51894/0 (socket says 172.21.15.50:51894) 2023-10-19T10:37:01.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daeffd640 1 -- 172.21.15.50:0/1601105068 learned_addr learned my addr 172.21.15.50:0/1601105068 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:01.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5dae7fc640 1 -- 172.21.15.50:0/1601105068 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788727701 0 0) 0x7f5db006aa30 con 0x7f5db015c950 2023-10-19T10:37:01.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5dae7fc640 1 -- 172.21.15.50:0/1601105068 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d98003520 con 0x7f5db015c950 2023-10-19T10:37:01.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daffff640 1 -- 172.21.15.50:0/1601105068 >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 msgr2=0x7f5db00d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:01.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daffff640 1 --2- 172.21.15.50:0/1601105068 >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 0x7f5db00d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daffff640 1 -- 172.21.15.50:0/1601105068 >> v1:172.21.15.183:6789/0 conn(0x7f5db015c950 legacy=0x7f5db006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daffff640 1 -- 172.21.15.50:0/1601105068 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db006ac80 con 0x7f5db00cd920 2023-10-19T10:37:01.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.788+0000 7f5daf7fe640 1 --2- 172.21.15.50:0/1601105068 >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 0x7f5db00d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:01.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5daffff640 1 --2- 172.21.15.50:0/1601105068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db00d26a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009d90 tx=0x7f5da802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e721b2f110067d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5dae7fc640 1 -- 172.21.15.50:0/1601105068 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da802cbc0 con 0x7f5db00cd920 2023-10-19T10:37:01.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5dae7fc640 1 -- 172.21.15.50:0/1601105068 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da802cd20 con 0x7f5db00cd920 2023-10-19T10:37:01.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5dae7fc640 1 -- 172.21.15.50:0/1601105068 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8034640 con 0x7f5db00cd920 2023-10-19T10:37:01.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1601105068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 msgr2=0x7f5db00d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1601105068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db00d26a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009d90 tx=0x7f5da802be10 comp rx=0 tx=0).stop 2023-10-19T10:37:01.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1601105068 shutdown_connections 2023-10-19T10:37:01.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1601105068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db00d26a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1601105068 >> v2:172.21.15.183:3300/0 conn(0x7f5db015c570 0x7f5db00d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1601105068 >> 172.21.15.50:0/1601105068 conn(0x7f5db0089ff0 msgr2=0x7f5db009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1601105068 shutdown_connections 2023-10-19T10:37:01.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.789+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1601105068 wait complete. 2023-10-19T10:37:01.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 Processor -- start 2023-10-19T10:37:01.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 -- start start 2023-10-19T10:37:01.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 0x7f5db014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db015c950 0x7f5db015e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db015f670 con 0x7f5db015c570 2023-10-19T10:37:01.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db015e9f0 con 0x7f5db015c950 2023-10-19T10:37:01.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.791+0000 7f5db5cb1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db015ecd0 con 0x7f5db00cd920 2023-10-19T10:37:01.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 0x7f5db014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db015c950 0x7f5db015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 0x7f5db014fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39604/0 (socket says 172.21.15.50:39604) 2023-10-19T10:37:01.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 -- 172.21.15.50:0/1089686079 learned_addr learned my addr 172.21.15.50:0/1089686079 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:01.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daf7fe640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 -- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 msgr2=0x7f5db015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 -- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db015c950 msgr2=0x7f5db015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db015c950 0x7f5db015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 -- 172.21.15.50:0/1089686079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00093a0 con 0x7f5db015c570 2023-10-19T10:37:01.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daffff640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db015c950 0x7f5db015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:01.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5daeffd640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 0x7f5db014fd20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c500 tx=0x7f5da400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4005ce0 con 0x7f5db015c570 2023-10-19T10:37:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da4005e40 con 0x7f5db015c570 2023-10-19T10:37:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da8009a70 con 0x7f5db015c570 2023-10-19T10:37:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4007330 con 0x7f5db015c570 2023-10-19T10:37:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.792+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db015f290 con 0x7f5db015c570 2023-10-19T10:37:01.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.793+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db0073990 con 0x7f5db015c570 2023-10-19T10:37:01.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.794+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5da4007490 con 0x7f5db015c570 2023-10-19T10:37:01.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.794+0000 7f5d97fff640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9c06a050 0x7f5d9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:01.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.794+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5da4087de0 con 0x7f5db015c570 2023-10-19T10:37:01.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.794+0000 7f5daf7fe640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9c06a050 0x7f5d9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:01.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.795+0000 7f5daf7fe640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9c06a050 0x7f5d9c06c510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009cc0 tx=0x7f5da0009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:01.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.798+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5da40533e0 con 0x7f5db015c570 2023-10-19T10:37:01.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.974+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5db00d3480 con 0x7f5db015c570 2023-10-19T10:37:01.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.975+0000 7f5d97fff640 1 -- 172.21.15.50:0/1089686079 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f5da4017020 con 0x7f5db015c570 2023-10-19T10:37:01.978 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:01.978 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:01.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9c06a050 msgr2=0x7f5d9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9c06a050 0x7f5d9c06c510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009cc0 tx=0x7f5da0009110 comp rx=0 tx=0).stop 2023-10-19T10:37:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 msgr2=0x7f5db014fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 0x7f5db014fd20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c500 tx=0x7f5da400c9c0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 shutdown_connections 2023-10-19T10:37:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9c06a050 0x7f5d9c06c510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db015c950 0x7f5db015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db015c570 0x7f5db014fd20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 --2- 172.21.15.50:0/1089686079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db00cd920 0x7f5db015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 >> 172.21.15.50:0/1089686079 conn(0x7f5db0089ff0 msgr2=0x7f5db015bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 shutdown_connections 2023-10-19T10:37:01.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:01.977+0000 7f5db5cb1640 1 -- 172.21.15.50:0/1089686079 wait complete. 2023-10-19T10:37:02.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-19T10:36:54.662444+0000', 'modified': '2023-10-19T10:37:00.329659+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6837/4204689263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6835', 'nonce': 4204689263}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 4204689263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-19T10:37:02.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-19T10:37:02.393 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-19T10:37:02.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.502+0000 7fad7a40b640 1 Processor -- start 2023-10-19T10:37:02.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.502+0000 7fad7a40b640 1 -- start start 2023-10-19T10:37:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad7a40b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fad740b2400 0x7fad7415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad7a40b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fad74160100 0x7fad74160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad7a40b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad741610a0 con 0x7fad740ce3c0 2023-10-19T10:37:02.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad7a40b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7415cad0 con 0x7fad74160100 2023-10-19T10:37:02.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad7a40b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740ce7a0 con 0x7fad740b2400 2023-10-19T10:37:02.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad789a8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fad74160100 0x7fad74160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad73fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fad740ce3c0 0x7fad740b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57076/0 (socket says 172.21.15.50:57076) 2023-10-19T10:37:02.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.503+0000 7fad73fff640 1 -- 172.21.15.50:0/2747689754 learned_addr learned my addr 172.21.15.50:0/2747689754 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:02.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad789a8640 1 -- 172.21.15.50:0/2747689754 >> v2:172.21.15.183:3300/0 conn(0x7fad740b2400 msgr2=0x7fad7415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad737fe640 1 --2- 172.21.15.50:0/2747689754 >> v2:172.21.15.183:3300/0 conn(0x7fad740b2400 0x7fad7415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad789a8640 1 --2- 172.21.15.50:0/2747689754 >> v2:172.21.15.183:3300/0 conn(0x7fad740b2400 0x7fad7415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad789a8640 1 -- 172.21.15.50:0/2747689754 >> v1:172.21.15.50:6789/0 conn(0x7fad740ce3c0 legacy=0x7fad740b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad789a8640 1 -- 172.21.15.50:0/2747689754 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74160dc0 con 0x7fad74160100 2023-10-19T10:37:02.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad72ffd640 1 -- 172.21.15.50:0/2747689754 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247203344 0 0) 0x7fad741610a0 con 0x7fad740ce3c0 2023-10-19T10:37:02.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad737fe640 1 --2- 172.21.15.50:0/2747689754 >> v2:172.21.15.183:3300/0 conn(0x7fad740b2400 0x7fad7415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:02.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.504+0000 7fad789a8640 1 --2- 172.21.15.50:0/2747689754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad74160100 0x7fad74160510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad74145cb0 tx=0x7fad6c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9a9ff8ae8f08ff server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad72ffd640 1 -- 172.21.15.50:0/2747689754 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02cc80 con 0x7fad74160100 2023-10-19T10:37:02.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad72ffd640 1 -- 172.21.15.50:0/2747689754 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c02cde0 con 0x7fad74160100 2023-10-19T10:37:02.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad72ffd640 1 -- 172.21.15.50:0/2747689754 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c034660 con 0x7fad74160100 2023-10-19T10:37:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 -- 172.21.15.50:0/2747689754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad74160100 msgr2=0x7fad74160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 --2- 172.21.15.50:0/2747689754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad74160100 0x7fad74160510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad74145cb0 tx=0x7fad6c02bed0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 -- 172.21.15.50:0/2747689754 shutdown_connections 2023-10-19T10:37:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 --2- 172.21.15.50:0/2747689754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad74160100 0x7fad74160510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 --2- 172.21.15.50:0/2747689754 >> v2:172.21.15.183:3300/0 conn(0x7fad740b2400 0x7fad7415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 -- 172.21.15.50:0/2747689754 >> 172.21.15.50:0/2747689754 conn(0x7fad7406f3f0 msgr2=0x7fad7406f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 -- 172.21.15.50:0/2747689754 shutdown_connections 2023-10-19T10:37:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.505+0000 7fad7a40b640 1 -- 172.21.15.50:0/2747689754 wait complete. 2023-10-19T10:37:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.506+0000 7fad7a40b640 1 Processor -- start 2023-10-19T10:37:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.506+0000 7fad7a40b640 1 -- start start 2023-10-19T10:37:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad7a40b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad740b2400 0x7fad74159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad7a40b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 0x7fad7415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad7a40b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad74160100 0x7fad740c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad7a40b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad740b37f0 con 0x7fad740ce3c0 2023-10-19T10:37:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad7a40b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad740b2ec0 con 0x7fad74160100 2023-10-19T10:37:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad7a40b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad740b31a0 con 0x7fad740b2400 2023-10-19T10:37:02.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad737fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 0x7fad7415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad737fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 0x7fad7415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39620/0 (socket says 172.21.15.50:39620) 2023-10-19T10:37:02.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.507+0000 7fad737fe640 1 -- 172.21.15.50:0/1661028006 learned_addr learned my addr 172.21.15.50:0/1661028006 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:02.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad737fe640 1 -- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad740b2400 msgr2=0x7fad74159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad737fe640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad740b2400 0x7fad74159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad737fe640 1 -- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad74160100 msgr2=0x7fad740c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad789a8640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad74160100 0x7fad740c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad737fe640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad74160100 0x7fad740c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad737fe640 1 -- 172.21.15.50:0/1661028006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c009a70 con 0x7fad740ce3c0 2023-10-19T10:37:02.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad737fe640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 0x7fad7415b310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c7f0 tx=0x7fad6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad70ff9640 1 -- 172.21.15.50:0/1661028006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68007a30 con 0x7fad740ce3c0 2023-10-19T10:37:02.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad70ff9640 1 -- 172.21.15.50:0/1661028006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68007b90 con 0x7fad740ce3c0 2023-10-19T10:37:02.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad70ff9640 1 -- 172.21.15.50:0/1661028006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800ce40 con 0x7fad740ce3c0 2023-10-19T10:37:02.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad740b3480 con 0x7fad740ce3c0 2023-10-19T10:37:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.508+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad740c9570 con 0x7fad740ce3c0 2023-10-19T10:37:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.509+0000 7fad70ff9640 1 -- 172.21.15.50:0/1661028006 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fad680113f0 con 0x7fad740ce3c0 2023-10-19T10:37:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.510+0000 7fad70ff9640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad6006a050 0x7fad6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.510+0000 7fad70ff9640 1 -- 172.21.15.50:0/1661028006 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fad68088a40 con 0x7fad740ce3c0 2023-10-19T10:37:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.510+0000 7fad73fff640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad6006a050 0x7fad6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.511+0000 7fad73fff640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad6006a050 0x7fad6006c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad64007fd0 tx=0x7fad64007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.511+0000 7fad7cf7c740 1 Processor -- start 2023-10-19T10:37:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.511+0000 7fad7cf7c740 1 -- start start 2023-10-19T10:37:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 0x56128849c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849fd50 0x5612884a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884cba90 0x5612884cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128840e060 con 0x56128849bcc0 2023-10-19T10:37:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128840b750 con 0x5612884cba90 2023-10-19T10:37:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x561288407040 con 0x56128849fd50 2023-10-19T10:37:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 0x56128849c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 0x56128849c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39632/0 (socket says 172.21.15.50:39632) 2023-10-19T10:37:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 -- 172.21.15.50:0/3070346616 learned_addr learned my addr 172.21.15.50:0/3070346616 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad737fe640 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849fd50 0x5612884a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad789a8640 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884cba90 0x5612884cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 -- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849fd50 msgr2=0x5612884a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849fd50 0x5612884a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 -- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884cba90 msgr2=0x5612884cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884cba90 0x5612884cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 -- 172.21.15.50:0/3070346616 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128840bcb0 con 0x56128849bcc0 2023-10-19T10:37:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad789a8640 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884cba90 0x5612884cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:02.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad73fff640 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 0x56128849c0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad64007580 tx=0x7fad6400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e94b9f412275634 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad59ffb640 1 -- 172.21.15.50:0/3070346616 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6401b070 con 0x56128849bcc0 2023-10-19T10:37:02.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad59ffb640 1 -- 172.21.15.50:0/3070346616 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad64007670 con 0x56128849bcc0 2023-10-19T10:37:02.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.512+0000 7fad59ffb640 1 -- 172.21.15.50:0/3070346616 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64005ea0 con 0x56128849bcc0 2023-10-19T10:37:02.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 -- 172.21.15.50:0/3070346616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 msgr2=0x56128849c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 0x56128849c0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad64007580 tx=0x7fad6400f000 comp rx=0 tx=0).stop 2023-10-19T10:37:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 -- 172.21.15.50:0/3070346616 shutdown_connections 2023-10-19T10:37:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884cba90 0x5612884cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849fd50 0x5612884a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/3070346616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56128849bcc0 0x56128849c0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 -- 172.21.15.50:0/3070346616 >> 172.21.15.50:0/3070346616 conn(0x56128849dd80 msgr2=0x56128849f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 -- 172.21.15.50:0/3070346616 shutdown_connections 2023-10-19T10:37:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.513+0000 7fad7cf7c740 1 -- 172.21.15.50:0/3070346616 wait complete. 2023-10-19T10:37:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.514+0000 7fad7cf7c740 1 -- start start 2023-10-19T10:37:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.514+0000 7fad7cf7c740 1 Processor -- start 2023-10-19T10:37:02.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5612884e5910 0x5612884bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612883c3d00 con 0x5612884cba90 2023-10-19T10:37:02.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612883c39a0 con 0x5612884b90d0 2023-10-19T10:37:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612883891f0 con 0x5612884e5910 2023-10-19T10:37:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 10 client.0 _ll_register_callbacks cb 0x5612883fed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-19T10:37:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39638/0 (socket says 172.21.15.50:39638) 2023-10-19T10:37:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 -- 172.21.15.50:0/1645603832 learned_addr learned my addr 172.21.15.50:0/1645603832 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad737fe640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad789a8640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5612884e5910 0x5612884bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 -- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5612884e5910 msgr2=0x5612884bb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5612884e5910 0x5612884bb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 -- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 msgr2=0x5612884bb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884bb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 -- 172.21.15.50:0/1645603832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad64007230 con 0x5612884cba90 2023-10-19T10:37:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad73fff640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad64007f70 tx=0x7fad640010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b469c04843f974b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2023-10-19T10:37:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad52ffd640 1 -- 172.21.15.50:0/1645603832 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64025070 con 0x5612884cba90 2023-10-19T10:37:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad52ffd640 1 -- 172.21.15.50:0/1645603832 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fad64012440 con 0x5612884cba90 2023-10-19T10:37:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612883dbe00 con 0x5612884cba90 2023-10-19T10:37:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad52ffd640 1 -- 172.21.15.50:0/1645603832 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad64012600 con 0x5612884cba90 2023-10-19T10:37:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 10 client.4458 fetch_fsmap learned FSMap version 7 2023-10-19T10:37:02.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.515+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5612884056c0 con 0x5612884cba90 2023-10-19T10:37:02.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad52ffd640 1 -- 172.21.15.50:0/1645603832 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612883dbe00 con 0x5612884cba90 2023-10-19T10:37:02.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad52ffd640 1 -- 172.21.15.50:0/1645603832 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fad64012760 con 0x5612884cba90 2023-10-19T10:37:02.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad7cf7c740 10 client.4458 fetch_fsmap finished waiting for FSMap version 7 2023-10-19T10:37:02.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad7cf7c740 10 client.4458 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-19T10:37:02.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad7cf7c740 20 client.4458 populate_metadata read hostname 'smithi050' 2023-10-19T10:37:02.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884c9f20 0x5612884e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad7cf7c740 4 client.4458 mds_command: new command op to 4285 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-19T10:37:02.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 --> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561288400020 con 0x5612884c9f20 2023-10-19T10:37:02.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad737fe640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884c9f20 0x5612884e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad52ffd640 10 client.4458 ms_handle_connect on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:02.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.516+0000 7fad737fe640 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884c9f20 0x5612884e9fb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2e9cc0dfb16788 server_cookie=7132c0eb3165c149 in_seq=0 out_seq=0 2023-10-19T10:37:02.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad52ffd640 1 -- 172.21.15.50:0/1645603832 <== mds.0 v2:172.21.15.183:6835/4204689263 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561288400020 con 0x5612884c9f20 2023-10-19T10:37:02.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad52ffd640 10 client.4458 handle_command_reply: tid=0 2023-10-19T10:37:02.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 client.4458 shutdown 2023-10-19T10:37:02.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 msgr2=0x5612884e6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad64007f70 tx=0x7fad640010f0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 shutdown_connections 2023-10-19T10:37:02.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884c9f20 0x5612884e9fb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5612884e5910 0x5612884bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/1645603832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad52ffd640 0 client.4458 ms_handle_reset on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:02.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 >> 172.21.15.50:0/1645603832 conn(0x56128849bcc0 msgr2=0x5612884e7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 shutdown_connections 2023-10-19T10:37:02.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.517+0000 7fad7cf7c740 1 -- 172.21.15.50:0/1645603832 wait complete. 2023-10-19T10:37:02.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.518+0000 7fad7cf7c740 20 client.4458 trim_cache size 0 max 16384 2023-10-19T10:37:02.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 Processor -- start 2023-10-19T10:37:02.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 -- start start 2023-10-19T10:37:02.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612884bf020 con 0x5612884cba90 2023-10-19T10:37:02.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128849aa90 con 0x5612884b90d0 2023-10-19T10:37:02.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.533+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612884c0400 con 0x56128849bcc0 2023-10-19T10:37:02.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39650/0 (socket says 172.21.15.50:39650) 2023-10-19T10:37:02.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 -- 172.21.15.50:0/2421691782 learned_addr learned my addr 172.21.15.50:0/2421691782 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:02.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 -- 172.21.15.50:0/2421691782 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 msgr2=0x5612884cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 -- 172.21.15.50:0/2421691782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 msgr2=0x5612884e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 -- 172.21.15.50:0/2421691782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612883a5c20 con 0x5612884cba90 2023-10-19T10:37:02.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad789a8640 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e49a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009e80 tx=0x7fad6c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50c1a884716effb server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad5a7fc640 1 -- 172.21.15.50:0/2421691782 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003b90 con 0x5612884cba90 2023-10-19T10:37:02.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad5a7fc640 1 -- 172.21.15.50:0/2421691782 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c003cf0 con 0x5612884cba90 2023-10-19T10:37:02.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad5a7fc640 1 -- 172.21.15.50:0/2421691782 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004260 con 0x5612884cba90 2023-10-19T10:37:02.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad7cf7c740 1 -- 172.21.15.50:0/2421691782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 msgr2=0x5612884e49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.534+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e49a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009e80 tx=0x7fad6c02e000 comp rx=0 tx=0).stop 2023-10-19T10:37:02.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 -- 172.21.15.50:0/2421691782 shutdown_connections 2023-10-19T10:37:02.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e49a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/2421691782 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 -- 172.21.15.50:0/2421691782 >> 172.21.15.50:0/2421691782 conn(0x561288497180 msgr2=0x5612884bd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 -- 172.21.15.50:0/2421691782 shutdown_connections 2023-10-19T10:37:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 -- 172.21.15.50:0/2421691782 wait complete. 2023-10-19T10:37:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 -- start start 2023-10-19T10:37:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.535+0000 7fad7cf7c740 1 Processor -- start 2023-10-19T10:37:02.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612884c8320 con 0x5612884cba90 2023-10-19T10:37:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612884ce660 con 0x5612884b90d0 2023-10-19T10:37:02.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x561288549e80 con 0x56128849bcc0 2023-10-19T10:37:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad7cf7c740 10 client.0 _ll_register_callbacks cb 0x7fad64001390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-19T10:37:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad789a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad737fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad73fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad789a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39662/0 (socket says 172.21.15.50:39662) 2023-10-19T10:37:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad737fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:51964/0 (socket says 172.21.15.50:51964) 2023-10-19T10:37:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad737fe640 1 -- 172.21.15.50:0/4044819008 learned_addr learned my addr 172.21.15.50:0/4044819008 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad73fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:53688/0 (socket says 172.21.15.50:53688) 2023-10-19T10:37:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad789a8640 1 -- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 msgr2=0x5612884f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.536+0000 7fad789a8640 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad789a8640 1 -- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 msgr2=0x5612884f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad789a8640 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad789a8640 1 -- 172.21.15.50:0/4044819008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad74160dc0 con 0x5612884cba90 2023-10-19T10:37:02.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad789a8640 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad6c042000 tx=0x7fad6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155671ea674844d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2023-10-19T10:37:02.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad52ffd640 1 -- 172.21.15.50:0/4044819008 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c04f070 con 0x5612884cba90 2023-10-19T10:37:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad64007670 con 0x5612884cba90 2023-10-19T10:37:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad52ffd640 1 -- 172.21.15.50:0/4044819008 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fad6c04a9c0 con 0x5612884cba90 2023-10-19T10:37:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad52ffd640 1 -- 172.21.15.50:0/4044819008 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad6c007080 con 0x5612884cba90 2023-10-19T10:37:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 10 client.4464 fetch_fsmap learned FSMap version 7 2023-10-19T10:37:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5612883cb1f0 con 0x5612884cba90 2023-10-19T10:37:02.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad52ffd640 1 -- 172.21.15.50:0/4044819008 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad64007670 con 0x5612884cba90 2023-10-19T10:37:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad52ffd640 1 -- 172.21.15.50:0/4044819008 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fad6c04ab80 con 0x5612884cba90 2023-10-19T10:37:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 10 client.4464 fetch_fsmap finished waiting for FSMap version 7 2023-10-19T10:37:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 10 client.4464 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-19T10:37:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 20 client.4464 populate_metadata read hostname 'smithi050' 2023-10-19T10:37:02.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884e5910 0x5612884e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 4 client.4464 mds_command: new command op to 4285 tid=0[{"prefix": "status"}] 2023-10-19T10:37:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 --> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5612883ffd40 con 0x5612884e5910 2023-10-19T10:37:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.537+0000 7fad73fff640 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884e5910 0x5612884e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad73fff640 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884e5910 0x5612884e81e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f3875662a7bf37 server_cookie=83742d4c72c58fff in_seq=0 out_seq=0 2023-10-19T10:37:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad52ffd640 10 client.4464 ms_handle_connect on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:02.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad52ffd640 1 -- 172.21.15.50:0/4044819008 <== mds.0 v2:172.21.15.183:6835/4204689263 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5612883ffd40 con 0x5612884e5910 2023-10-19T10:37:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad52ffd640 10 client.4464 handle_command_reply: tid=0 2023-10-19T10:37:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 client.4464 shutdown 2023-10-19T10:37:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 msgr2=0x5612884e6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad6c042000 tx=0x7fad6c004ab0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 shutdown_connections 2023-10-19T10:37:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5612884e5910 0x5612884e81e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612884cba90 0x5612884e6360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612884b90d0 0x5612884f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad52ffd640 0 client.4464 ms_handle_reset on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:02.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 --2- 172.21.15.50:0/4044819008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x56128849bcc0 0x5612884f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 >> 172.21.15.50:0/4044819008 conn(0x561288497180 msgr2=0x5612884a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 shutdown_connections 2023-10-19T10:37:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.538+0000 7fad7cf7c740 1 -- 172.21.15.50:0/4044819008 wait complete. 2023-10-19T10:37:02.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7cf7c740 20 client.4464 trim_cache size 0 max 16384 2023-10-19T10:37:02.569 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:37:02.569 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "2b47c9ed-f967-49ff-9d46-d95ee8c39a83", 2023-10-19T10:37:02.569 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2023-10-19T10:37:02.569 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4285, 2023-10-19T10:37:02.570 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2023-10-19T10:37:02.570 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2023-10-19T10:37:02.570 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2023-10-19T10:37:02.570 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 7.2379763549999998, 2023-10-19T10:37:02.570 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 7, 2023-10-19T10:37:02.570 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 27, 2023-10-19T10:37:02.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 27, 2023-10-19T10:37:02.571 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 12.267261437 2023-10-19T10:37:02.571 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:37:02.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad6006a050 msgr2=0x7fad6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad6006a050 0x7fad6006c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad64007fd0 tx=0x7fad64007920 comp rx=0 tx=0).stop 2023-10-19T10:37:02.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 msgr2=0x7fad7415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 0x7fad7415b310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c7f0 tx=0x7fad6800cbf0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 shutdown_connections 2023-10-19T10:37:02.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad6006a050 0x7fad6006c510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad74160100 0x7fad740c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad740ce3c0 0x7fad7415b310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 --2- 172.21.15.50:0/1661028006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad740b2400 0x7fad74159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 >> 172.21.15.50:0/1661028006 conn(0x7fad7406f3f0 msgr2=0x7fad7406c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 shutdown_connections 2023-10-19T10:37:02.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.539+0000 7fad7a40b640 1 -- 172.21.15.50:0/1661028006 wait complete. 2023-10-19T10:37:02.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-19T10:37:02.576 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:02.577 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:37:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.680+0000 7fa9b278b640 1 Processor -- start 2023-10-19T10:37:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.680+0000 7fa9b278b640 1 -- start start 2023-10-19T10:37:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9b278b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa9ac0d0e30 0x7fa9ac15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9b278b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9ac0cd920 0x7fa9ac0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9b278b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac06aa30 con 0x7fa9ac15c570 2023-10-19T10:37:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9b278b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac15ce90 con 0x7fa9ac0cd920 2023-10-19T10:37:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9b278b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac06b2d0 con 0x7fa9ac0d0e30 2023-10-19T10:37:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9b0d28640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9ac0cd920 0x7fa9ac0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9abfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa9ac15c570 0x7fa9ac06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57088/0 (socket says 172.21.15.50:57088) 2023-10-19T10:37:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.681+0000 7fa9abfff640 1 -- 172.21.15.50:0/1273652459 learned_addr learned my addr 172.21.15.50:0/1273652459 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9ab7fe640 1 --2- 172.21.15.50:0/1273652459 >> v2:172.21.15.183:3300/0 conn(0x7fa9ac0d0e30 0x7fa9ac15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1273652459 >> v2:172.21.15.183:3300/0 conn(0x7fa9ac0d0e30 msgr2=0x7fa9ac15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9b0d28640 1 --2- 172.21.15.50:0/1273652459 >> v2:172.21.15.183:3300/0 conn(0x7fa9ac0d0e30 0x7fa9ac15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1273652459 >> v1:172.21.15.50:6789/0 conn(0x7fa9ac15c570 legacy=0x7fa9ac06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9aaffd640 1 -- 172.21.15.50:0/1273652459 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896756103 0 0) 0x7fa9ac06aa30 con 0x7fa9ac15c570 2023-10-19T10:37:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1273652459 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0d1270 con 0x7fa9ac0cd920 2023-10-19T10:37:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.682+0000 7fa9b0d28640 1 --2- 172.21.15.50:0/1273652459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac0cd920 0x7fa9ac0d26a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0b87c0 tx=0x7fa9a402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e117422ff79b2940 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9aaffd640 1 -- 172.21.15.50:0/1273652459 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a402cbf0 con 0x7fa9ac0cd920 2023-10-19T10:37:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9aaffd640 1 -- 172.21.15.50:0/1273652459 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a402cd50 con 0x7fa9ac0cd920 2023-10-19T10:37:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9aaffd640 1 -- 172.21.15.50:0/1273652459 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4034660 con 0x7fa9ac0cd920 2023-10-19T10:37:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 -- 172.21.15.50:0/1273652459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac0cd920 msgr2=0x7fa9ac0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1273652459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac0cd920 0x7fa9ac0d26a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0b87c0 tx=0x7fa9a402be40 comp rx=0 tx=0).stop 2023-10-19T10:37:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 -- 172.21.15.50:0/1273652459 shutdown_connections 2023-10-19T10:37:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1273652459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac0cd920 0x7fa9ac0d26a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1273652459 >> v2:172.21.15.183:3300/0 conn(0x7fa9ac0d0e30 0x7fa9ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 -- 172.21.15.50:0/1273652459 >> 172.21.15.50:0/1273652459 conn(0x7fa9ac089ff0 msgr2=0x7fa9ac09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 -- 172.21.15.50:0/1273652459 shutdown_connections 2023-10-19T10:37:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.683+0000 7fa9b278b640 1 -- 172.21.15.50:0/1273652459 wait complete. 2023-10-19T10:37:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.684+0000 7fa9b278b640 1 Processor -- start 2023-10-19T10:37:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.684+0000 7fa9b278b640 1 -- start start 2023-10-19T10:37:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b278b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9ac0cd920 0x7fa9ac156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b278b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ac0d0e30 0x7fa9ac0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b278b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 0x7fa9ac0c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b278b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac15a960 con 0x7fa9ac15c570 2023-10-19T10:37:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b278b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c6c00 con 0x7fa9ac0d0e30 2023-10-19T10:37:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b278b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c6df0 con 0x7fa9ac0cd920 2023-10-19T10:37:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b0d28640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 0x7fa9ac0c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b0d28640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 0x7fa9ac0c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39676/0 (socket says 172.21.15.50:39676) 2023-10-19T10:37:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1770218359 learned_addr learned my addr 172.21.15.50:0/1770218359 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9ac0cd920 msgr2=0x7fa9ac156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.685+0000 7fa9b0d28640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9ac0cd920 0x7fa9ac156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ac0d0e30 msgr2=0x7fa9ac0c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9ab7fe640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ac0d0e30 0x7fa9ac0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9b0d28640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ac0d0e30 0x7fa9ac0c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9b0d28640 1 -- 172.21.15.50:0/1770218359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009a70 con 0x7fa9ac15c570 2023-10-19T10:37:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9b0d28640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 0x7fa9ac0c8040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009e80 tx=0x7fa9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4040650 con 0x7fa9ac15c570 2023-10-19T10:37:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40407b0 con 0x7fa9ac15c570 2023-10-19T10:37:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac0c70d0 con 0x7fa9ac15c570 2023-10-19T10:37:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4040b60 con 0x7fa9ac15c570 2023-10-19T10:37:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.686+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac0c7690 con 0x7fa9ac15c570 2023-10-19T10:37:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.687+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac073990 con 0x7fa9ac15c570 2023-10-19T10:37:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.687+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9a4040cc0 con 0x7fa9ac15c570 2023-10-19T10:37:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.688+0000 7fa9a8ff9640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa99806a050 0x7fa99806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.688+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa9a40ab620 con 0x7fa9ac15c570 2023-10-19T10:37:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.688+0000 7fa9abfff640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa99806a050 0x7fa99806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.688+0000 7fa9abfff640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa99806a050 0x7fa99806c510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa99c007fd0 tx=0x7fa99c0077c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.692+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa9a4076a20 con 0x7fa9ac15c570 2023-10-19T10:37:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.902+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9ac0d3480 con 0x7fa9ac15c570 2023-10-19T10:37:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.902+0000 7fa9a8ff9640 1 -- 172.21.15.50:0/1770218359 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa9a4009450 con 0x7fa9ac15c570 2023-10-19T10:37:02.905 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-19T10:37:02.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa99806a050 msgr2=0x7fa99806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa99806a050 0x7fa99806c510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa99c007fd0 tx=0x7fa99c0077c0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 msgr2=0x7fa9ac0c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:02.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 0x7fa9ac0c8040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009e80 tx=0x7fa9a4004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:02.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 shutdown_connections 2023-10-19T10:37:02.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa99806a050 0x7fa99806c510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9ac15c570 0x7fa9ac0c8040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ac0d0e30 0x7fa9ac0c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 --2- 172.21.15.50:0/1770218359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9ac0cd920 0x7fa9ac156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:02.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 >> 172.21.15.50:0/1770218359 conn(0x7fa9ac089ff0 msgr2=0x7fa9ac0ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:02.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 shutdown_connections 2023-10-19T10:37:02.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:02.904+0000 7fa9b278b640 1 -- 172.21.15.50:0/1770218359 wait complete. 2023-10-19T10:37:02.917 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-19T10:37:02.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:37:02.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-19T10:37:02.918 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-19T10:37:02.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:02.918 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-19T10:37:02.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:37:02.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:37:02.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-19T10:37:02.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:02.922 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-19T10:37:02.938 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:02.938 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:02.938 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:02.938 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:02.938 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:02.939 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:02.939 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:02.939 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:02.939 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2023-10-19T10:37:02.940 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:02.940 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:02.940 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2023-10-19T10:37:02.940 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:37:02.940 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:02.941 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2023-10-19T10:37:02.941 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-19T10:37:02.941 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42678sec preferred_lft 42678sec 2023-10-19T10:37:02.941 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2023-10-19T10:37:02.941 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:02.942 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:02.942 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:02.942 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2023-10-19T10:37:02.943 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-19T10:37:02.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:02.943 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:02.944 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:02.945 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2023-10-19T10:37:02.945 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2023-10-19T10:37:02.945 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2023-10-19T10:37:02.946 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-19T10:37:02.946 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:03.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.112 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:37:03.142 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-19T10:37:03.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.144 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-19T10:37:03.213 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-19T10:37:03.213 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:37:03.213 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-19T10:37:03.213 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-19T10:37:03.214 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:37:03.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.215 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:03.215 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:03.215 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-19T10:37:03.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-19T10:37:03.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-19T10:37:03.216 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:03.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.501 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:37:03.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.518 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2023-10-19T10:37:03.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.573 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:03.574 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:03.574 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-19T10:37:03.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-19T10:37:03.575 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:03.667 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-19T10:37:03.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.668 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:03.668 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:03.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-19T10:37:03.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-19T10:37:03.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-19T10:37:03.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-19T10:37:03.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-19T10:37:03.670 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:03.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:03.808 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:03.808 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:03.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2023-10-19T10:37:03.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2023-10-19T10:37:03.809 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:03.859 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-19T10:37:03.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:03.860 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:03.875 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-19T10:37:03.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:03.875 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:03.930 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:37:04.001 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:04.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:04.072 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:04.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:04.074 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:37:04.128 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-19T10:37:04.128 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-19T10:37:04.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:37:04.203 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:04.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:04.274 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:04.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:2023-10-19T10:37:04.272+0000 7f1eeec0f480 -1 init, newargv = 0x558356251eb0 newargc=15 2023-10-19T10:37:04.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[94800]: starting ceph client 2023-10-19T10:37:04.276 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:04.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:04.277 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:37:04.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[94800]: starting fuse 2023-10-19T10:37:04.296 INFO:teuthology.orchestra.run.smithi050.stdout:41 2023-10-19T10:37:04.296 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-19T10:37:04.297 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:37:04.298 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-19T10:37:04.298 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-19T10:37:04.299 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-19T10:37:04.299 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-19T10:37:04.299 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:04.300 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-19T10:37:04.300 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-19T10:37:04.301 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-19T10:37:04.301 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-19T10:37:04.302 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:04.302 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-19T10:37:04.303 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-19T10:37:04.303 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-19T10:37:04.304 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-19T10:37:04.304 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:04.305 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-19T10:37:04.305 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:37:04.306 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:37:04.307 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:37:04.307 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-19T10:37:04.308 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-19T10:37:04.309 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-19T10:37:04.309 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:37:04.310 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:04.311 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.admin")) 2023-10-19T10:37:04.311 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-19T10:37:04.421 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.admin.94800.asok 2023-10-19T10:37:04.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:37:04.432 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.94800.asok 2023-10-19T10:37:04.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:04.433 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.94800.asok status 2023-10-19T10:37:04.573 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:37:04.574 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-19T10:37:04.574 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:37:04.574 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:37:04.575 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "admin", 2023-10-19T10:37:04.575 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-19T10:37:04.575 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-19T10:37:04.575 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "94800", 2023-10-19T10:37:04.575 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-19T10:37:04.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:04.576 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-19T10:37:04.576 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-19T10:37:04.576 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4479, 2023-10-19T10:37:04.576 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-19T10:37:04.577 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-19T10:37:04.577 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-19T10:37:04.577 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 4479 2023-10-19T10:37:04.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:04.577 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:37:04.577 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:37:04.578 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:37:04.578 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1261165859 2023-10-19T10:37:04.578 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:37:04.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:04.578 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:37:04.579 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:37:04.579 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:37:04.579 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1261165859 2023-10-19T10:37:04.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:04.579 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.4479 192.168.0.1:0/1261165859", 2023-10-19T10:37:04.580 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/1261165859", 2023-10-19T10:37:04.580 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-19T10:37:04.580 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 7, 2023-10-19T10:37:04.580 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 27, 2023-10-19T10:37:04.580 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:37:04.581 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-19T10:37:04.581 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-19T10:37:04.581 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:37:04.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:04.582 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-19T10:37:04.583 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:04.599 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-19T10:37:04.599 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:04.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:04.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:04.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:04.671 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-19T10:37:04.750 INFO:teuthology.orchestra.run.smithi050.stdout:# file: . 2023-10-19T10:37:04.750 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-19T10:37:04.751 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:04.751 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:37:04.751 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:04.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:04.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.949+0000 7f2ff2258640 1 Processor -- start 2023-10-19T10:37:04.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.949+0000 7f2ff2258640 1 -- start start 2023-10-19T10:37:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2ff2258640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2fec0d0e30 0x7f2fec15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2ff2258640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 0x7f2fec0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2ff2258640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fec06aa30 con 0x7f2fec15c570 2023-10-19T10:37:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2ff2258640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec15ce90 con 0x7f2fec0d0e30 2023-10-19T10:37:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2ff2258640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec06b2d0 con 0x7f2fec0cd920 2023-10-19T10:37:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 0x7f2fec0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2feaffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2fec0d0e30 0x7f2fec15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 0x7f2fec0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:53704/0 (socket says 172.21.15.50:53704) 2023-10-19T10:37:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 -- 172.21.15.50:0/215157109 learned_addr learned my addr 172.21.15.50:0/215157109 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2feb7fe640 1 --1- 172.21.15.50:0/215157109 >> v1:172.21.15.183:6789/0 conn(0x7f2fec15c570 0x7f2fec06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:38284/0 (socket says 172.21.15.50:38284) 2023-10-19T10:37:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 -- 172.21.15.50:0/215157109 >> v2:172.21.15.183:3300/0 conn(0x7f2fec0d0e30 msgr2=0x7f2fec15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2fea7fc640 1 -- 172.21.15.50:0/215157109 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041073674 0 0) 0x7f2fec06aa30 con 0x7f2fec15c570 2023-10-19T10:37:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 --2- 172.21.15.50:0/215157109 >> v2:172.21.15.183:3300/0 conn(0x7f2fec0d0e30 0x7f2fec15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 -- 172.21.15.50:0/215157109 >> v1:172.21.15.183:6789/0 conn(0x7f2fec15c570 legacy=0x7f2fec06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.950+0000 7f2febfff640 1 -- 172.21.15.50:0/215157109 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec0d1270 con 0x7f2fec0cd920 2023-10-19T10:37:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.951+0000 7f2febfff640 1 --2- 172.21.15.50:0/215157109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 0x7f2fec0d26a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009e50 tx=0x7f2fe402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ca9454aa7642a34 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.951+0000 7f2fea7fc640 1 -- 172.21.15.50:0/215157109 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe402ca70 con 0x7f2fec0cd920 2023-10-19T10:37:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.951+0000 7f2fea7fc640 1 -- 172.21.15.50:0/215157109 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe402cbd0 con 0x7f2fec0cd920 2023-10-19T10:37:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.951+0000 7f2fea7fc640 1 -- 172.21.15.50:0/215157109 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40343f0 con 0x7f2fec0cd920 2023-10-19T10:37:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 -- 172.21.15.50:0/215157109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 msgr2=0x7f2fec0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 --2- 172.21.15.50:0/215157109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 0x7f2fec0d26a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009e50 tx=0x7f2fe402bed0 comp rx=0 tx=0).stop 2023-10-19T10:37:04.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 -- 172.21.15.50:0/215157109 shutdown_connections 2023-10-19T10:37:04.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 --2- 172.21.15.50:0/215157109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec0cd920 0x7f2fec0d26a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:04.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 --2- 172.21.15.50:0/215157109 >> v2:172.21.15.183:3300/0 conn(0x7f2fec0d0e30 0x7f2fec15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:04.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 -- 172.21.15.50:0/215157109 >> 172.21.15.50:0/215157109 conn(0x7f2fec089ff0 msgr2=0x7f2fec09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:04.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 -- 172.21.15.50:0/215157109 shutdown_connections 2023-10-19T10:37:04.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.952+0000 7f2ff2258640 1 -- 172.21.15.50:0/215157109 wait complete. 2023-10-19T10:37:04.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.953+0000 7f2ff2258640 1 Processor -- start 2023-10-19T10:37:04.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.953+0000 7f2ff2258640 1 -- start start 2023-10-19T10:37:04.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2ff2258640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fec0cd920 0x7f2fec154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:04.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2ff2258640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 0x7f2fec1f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:04.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2ff2258640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec15c570 0x7f2fec0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:04.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2ff2258640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0c8390 con 0x7f2fec0d0e30 2023-10-19T10:37:04.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2ff2258640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0c57a0 con 0x7f2fec0cd920 2023-10-19T10:37:04.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2ff2258640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0c7a60 con 0x7f2fec15c570 2023-10-19T10:37:04.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 0x7f2fec1f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:04.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2feaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 0x7f2fec1f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39686/0 (socket says 172.21.15.50:39686) 2023-10-19T10:37:04.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2feaffd640 1 -- 172.21.15.50:0/2437103933 learned_addr learned my addr 172.21.15.50:0/2437103933 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:04.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2feb7fe640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fec0cd920 0x7f2fec154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:04.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.954+0000 7f2febfff640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec15c570 0x7f2fec0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:04.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2feaffd640 1 -- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec15c570 msgr2=0x7f2fec0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:04.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2feaffd640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec15c570 0x7f2fec0c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:04.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2feaffd640 1 -- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fec0cd920 msgr2=0x7f2fec154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:04.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2feaffd640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fec0cd920 0x7f2fec154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:04.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2feaffd640 1 -- 172.21.15.50:0/2437103933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4009b30 con 0x7f2fec0d0e30 2023-10-19T10:37:04.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2feaffd640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 0x7f2fec1f5a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000e8a0 tx=0x7f2fe000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:04.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00094f0 con 0x7f2fec0d0e30 2023-10-19T10:37:04.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe000c940 con 0x7f2fec0d0e30 2023-10-19T10:37:04.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000ccf0 con 0x7f2fec0d0e30 2023-10-19T10:37:04.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec0c7d40 con 0x7f2fec0d0e30 2023-10-19T10:37:04.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.955+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec15d1a0 con 0x7f2fec0d0e30 2023-10-19T10:37:04.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.956+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec0c8540 con 0x7f2fec0d0e30 2023-10-19T10:37:04.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.956+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2fe0013470 con 0x7f2fec0d0e30 2023-10-19T10:37:04.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.957+0000 7f2fd3fff640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2fd806a050 0x7f2fd806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:04.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.957+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2fe00888a0 con 0x7f2fec0d0e30 2023-10-19T10:37:04.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.957+0000 7f2feb7fe640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2fd806a050 0x7f2fd806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:04.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.957+0000 7f2feb7fe640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2fd806a050 0x7f2fd806c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000fb0 tx=0x7f2fdc007b00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:04.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:04.961+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2fec0c8540 con 0x7f2fec0d0e30 2023-10-19T10:37:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.137+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fec0cead0 con 0x7f2fec0d0e30 2023-10-19T10:37:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.138+0000 7f2fd3fff640 1 -- 172.21.15.50:0/2437103933 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f2fe0053d30 con 0x7f2fec0d0e30 2023-10-19T10:37:05.141 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:05.141 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:05.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2fd806a050 msgr2=0x7f2fd806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2fd806a050 0x7f2fd806c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000fb0 tx=0x7f2fdc007b00 comp rx=0 tx=0).stop 2023-10-19T10:37:05.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 msgr2=0x7f2fec1f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 0x7f2fec1f5a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000e8a0 tx=0x7f2fe000ed60 comp rx=0 tx=0).stop 2023-10-19T10:37:05.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 shutdown_connections 2023-10-19T10:37:05.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2fd806a050 0x7f2fd806c510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2fec15c570 0x7f2fec0c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2fec0d0e30 0x7f2fec1f5a90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 --2- 172.21.15.50:0/2437103933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fec0cd920 0x7f2fec154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 >> 172.21.15.50:0/2437103933 conn(0x7f2fec089ff0 msgr2=0x7f2fec0ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 shutdown_connections 2023-10-19T10:37:05.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.140+0000 7f2ff2258640 1 -- 172.21.15.50:0/2437103933 wait complete. 2023-10-19T10:37:05.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:05.154 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.260+0000 7f5e0ba09640 1 Processor -- start 2023-10-19T10:37:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.260+0000 7f5e0ba09640 1 -- start start 2023-10-19T10:37:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e0ba09640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5e040b4070 0x7f5e040b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e0ba09640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e040d3480 con 0x7f5e0415cdf0 2023-10-19T10:37:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e0ba09640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e040b4990 con 0x7f5e0415c960 2023-10-19T10:37:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e0ba09640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0415d280 con 0x7f5e040b4070 2023-10-19T10:37:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e097a5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5e040b4070 0x7f5e040b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e09fa6640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5e0415cdf0 0x7f5e040cd4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57090/0 (socket says 172.21.15.50:57090) 2023-10-19T10:37:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e097a5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5e040b4070 0x7f5e040b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39698/0 (socket says 172.21.15.50:39698) 2023-10-19T10:37:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e09fa6640 1 -- 172.21.15.50:0/1872315739 learned_addr learned my addr 172.21.15.50:0/1872315739 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e097a5640 1 -- 172.21.15.50:0/1872315739 >> v1:172.21.15.183:6789/0 conn(0x7f5e0415c960 legacy=0x7f5e040d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e097a5640 1 -- 172.21.15.50:0/1872315739 >> v1:172.21.15.50:6789/0 conn(0x7f5e0415cdf0 legacy=0x7f5e040cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.261+0000 7f5e097a5640 1 -- 172.21.15.50:0/1872315739 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0415d560 con 0x7f5e040b4070 2023-10-19T10:37:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.262+0000 7f5e097a5640 1 --2- 172.21.15.50:0/1872315739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e040b4450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009790 tx=0x7f5e0002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a84da17a602b663 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.262+0000 7f5dfbfff640 1 -- 172.21.15.50:0/1872315739 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00030610 con 0x7f5e040b4070 2023-10-19T10:37:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.262+0000 7f5dfbfff640 1 -- 172.21.15.50:0/1872315739 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00030770 con 0x7f5e040b4070 2023-10-19T10:37:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.262+0000 7f5dfbfff640 1 -- 172.21.15.50:0/1872315739 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00030b00 con 0x7f5e040b4070 2023-10-19T10:37:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.262+0000 7f5e0ba09640 1 -- 172.21.15.50:0/1872315739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 msgr2=0x7f5e040b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.262+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/1872315739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e040b4450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009790 tx=0x7f5e0002b960 comp rx=0 tx=0).stop 2023-10-19T10:37:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.263+0000 7f5e0ba09640 1 -- 172.21.15.50:0/1872315739 shutdown_connections 2023-10-19T10:37:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.263+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/1872315739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e040b4450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.263+0000 7f5e0ba09640 1 -- 172.21.15.50:0/1872315739 >> 172.21.15.50:0/1872315739 conn(0x7f5e0409b970 msgr2=0x7f5e040d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.263+0000 7f5e0ba09640 1 -- 172.21.15.50:0/1872315739 shutdown_connections 2023-10-19T10:37:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.263+0000 7f5e0ba09640 1 -- 172.21.15.50:0/1872315739 wait complete. 2023-10-19T10:37:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.264+0000 7f5e0ba09640 1 Processor -- start 2023-10-19T10:37:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.264+0000 7f5e0ba09640 1 -- start start 2023-10-19T10:37:05.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e041583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0415c960 0x7f5e040c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e040c8810 0x7f5e040b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c9d20 con 0x7f5e040b4070 2023-10-19T10:37:05.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04158910 con 0x7f5e0415c960 2023-10-19T10:37:05.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c8d00 con 0x7f5e040c8810 2023-10-19T10:37:05.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e041583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e041583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39704/0 (socket says 172.21.15.50:39704) 2023-10-19T10:37:05.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 -- 172.21.15.50:0/3520696865 learned_addr learned my addr 172.21.15.50:0/3520696865 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:05.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e08fa4640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0415c960 0x7f5e040c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e09fa6640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e040c8810 0x7f5e040b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 -- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e040c8810 msgr2=0x7f5e040b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e040c8810 0x7f5e040b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 -- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0415c960 msgr2=0x7f5e040c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0415c960 0x7f5e040c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 -- 172.21.15.50:0/3520696865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000093a0 con 0x7f5e040b4070 2023-10-19T10:37:05.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e097a5640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e041583d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002bf70 tx=0x7f5e00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0003e040 con 0x7f5e040b4070 2023-10-19T10:37:05.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00039cb0 con 0x7f5e040b4070 2023-10-19T10:37:05.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00044630 con 0x7f5e040b4070 2023-10-19T10:37:05.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.265+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e040c96d0 con 0x7f5e040b4070 2023-10-19T10:37:05.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.266+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e040c8fe0 con 0x7f5e040b4070 2023-10-19T10:37:05.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.267+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040d09b0 con 0x7f5e040b4070 2023-10-19T10:37:05.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.267+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e00043390 con 0x7f5e040b4070 2023-10-19T10:37:05.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.267+0000 7f5df9ffb640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5de806a050 0x7f5de806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.267+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5e000aab20 con 0x7f5e040b4070 2023-10-19T10:37:05.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.267+0000 7f5e08fa4640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5de806a050 0x7f5de806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.268+0000 7f5e08fa4640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5de806a050 0x7f5de806c510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e0409fa10 tx=0x7f5df4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.272+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5e00075fb0 con 0x7f5e040b4070 2023-10-19T10:37:05.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.446+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e0404e420 con 0x7f5e040b4070 2023-10-19T10:37:05.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.447+0000 7f5df9ffb640 1 -- 172.21.15.50:0/3520696865 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e00034020 con 0x7f5e040b4070 2023-10-19T10:37:05.450 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:05.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5de806a050 msgr2=0x7f5de806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5de806a050 0x7f5de806c510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e0409fa10 tx=0x7f5df4009040 comp rx=0 tx=0).stop 2023-10-19T10:37:05.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 msgr2=0x7f5e041583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e041583d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002bf70 tx=0x7f5e00004730 comp rx=0 tx=0).stop 2023-10-19T10:37:05.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 shutdown_connections 2023-10-19T10:37:05.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5de806a050 0x7f5de806c510 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e040c8810 0x7f5e040b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0415c960 0x7f5e040c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 --2- 172.21.15.50:0/3520696865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e040b4070 0x7f5e041583d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.448+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 >> 172.21.15.50:0/3520696865 conn(0x7f5e0409b970 msgr2=0x7f5e0406a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.449+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 shutdown_connections 2023-10-19T10:37:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.449+0000 7f5e0ba09640 1 -- 172.21.15.50:0/3520696865 wait complete. 2023-10-19T10:37:05.462 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.568+0000 7ff2edf27640 1 Processor -- start 2023-10-19T10:37:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.568+0000 7ff2edf27640 1 -- start start 2023-10-19T10:37:05.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2edf27640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2e80b3090 0x7ff2e80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2edf27640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2e8161210 0x7ff2e80b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2edf27640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e80af250 con 0x7ff2e80b38f0 2023-10-19T10:37:05.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2edf27640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e80b0190 con 0x7ff2e80b3090 2023-10-19T10:37:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2edf27640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e80af510 con 0x7ff2e8161210 2023-10-19T10:37:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2e6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2e80b3090 0x7ff2e80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2e6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2e80b3090 0x7ff2e80d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39720/0 (socket says 172.21.15.50:39720) 2023-10-19T10:37:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2e6ffd640 1 -- 172.21.15.50:0/161336162 learned_addr learned my addr 172.21.15.50:0/161336162 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.569+0000 7ff2e7fff640 1 --2- 172.21.15.50:0/161336162 >> v2:172.21.15.183:3300/0 conn(0x7ff2e8161210 0x7ff2e80b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.570+0000 7ff2e6ffd640 1 -- 172.21.15.50:0/161336162 >> v2:172.21.15.183:3300/0 conn(0x7ff2e8161210 msgr2=0x7ff2e80b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.570+0000 7ff2e6ffd640 1 --2- 172.21.15.50:0/161336162 >> v2:172.21.15.183:3300/0 conn(0x7ff2e8161210 0x7ff2e80b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.570+0000 7ff2e6ffd640 1 -- 172.21.15.50:0/161336162 >> v1:172.21.15.183:6789/0 conn(0x7ff2e80b38f0 legacy=0x7ff2e8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.570+0000 7ff2e6ffd640 1 -- 172.21.15.50:0/161336162 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e80af7f0 con 0x7ff2e80b3090 2023-10-19T10:37:05.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.570+0000 7ff2e6ffd640 1 --2- 172.21.15.50:0/161336162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e80b3090 0x7ff2e80d30e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80b3470 tx=0x7ff2dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c8a790a1c20c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.570+0000 7ff2e67fc640 1 -- 172.21.15.50:0/161336162 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc031870 con 0x7ff2e80b3090 2023-10-19T10:37:05.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2e67fc640 1 -- 172.21.15.50:0/161336162 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc0319d0 con 0x7ff2e80b3090 2023-10-19T10:37:05.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2e67fc640 1 -- 172.21.15.50:0/161336162 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc031d60 con 0x7ff2e80b3090 2023-10-19T10:37:05.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 -- 172.21.15.50:0/161336162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e80b3090 msgr2=0x7ff2e80d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 --2- 172.21.15.50:0/161336162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e80b3090 0x7ff2e80d30e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80b3470 tx=0x7ff2dc02b910 comp rx=0 tx=0).stop 2023-10-19T10:37:05.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 -- 172.21.15.50:0/161336162 shutdown_connections 2023-10-19T10:37:05.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 --2- 172.21.15.50:0/161336162 >> v2:172.21.15.183:3300/0 conn(0x7ff2e8161210 0x7ff2e80b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 --2- 172.21.15.50:0/161336162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e80b3090 0x7ff2e80d30e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 -- 172.21.15.50:0/161336162 >> 172.21.15.50:0/161336162 conn(0x7ff2e809b960 msgr2=0x7ff2e80d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:05.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 -- 172.21.15.50:0/161336162 shutdown_connections 2023-10-19T10:37:05.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.571+0000 7ff2edf27640 1 -- 172.21.15.50:0/161336162 wait complete. 2023-10-19T10:37:05.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.572+0000 7ff2edf27640 1 Processor -- start 2023-10-19T10:37:05.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.572+0000 7ff2edf27640 1 -- start start 2023-10-19T10:37:05.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2edf27640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e80b3090 0x7ff2e8157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2edf27640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e80b38f0 0x7ff2e80ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2edf27640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 0x7ff2e80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2edf27640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e815a1d0 con 0x7ff2e8161210 2023-10-19T10:37:05.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2edf27640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e815a4b0 con 0x7ff2e80b38f0 2023-10-19T10:37:05.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2edf27640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e815a790 con 0x7ff2e80b3090 2023-10-19T10:37:05.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2e7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 0x7ff2e80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2e7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 0x7ff2e80cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39736/0 (socket says 172.21.15.50:39736) 2023-10-19T10:37:05.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2e7fff640 1 -- 172.21.15.50:0/2343432022 learned_addr learned my addr 172.21.15.50:0/2343432022 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:05.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2e77fe640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e80b3090 0x7ff2e8157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.573+0000 7ff2e6ffd640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e80b38f0 0x7ff2e80ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e7fff640 1 -- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e80b3090 msgr2=0x7ff2e8157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e7fff640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e80b3090 0x7ff2e8157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e7fff640 1 -- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e80b38f0 msgr2=0x7ff2e80ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e7fff640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e80b38f0 0x7ff2e80ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e7fff640 1 -- 172.21.15.50:0/2343432022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0094d0 con 0x7ff2e8161210 2023-10-19T10:37:05.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e6ffd640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e80b38f0 0x7ff2e80ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:05.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2e7fff640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 0x7ff2e80cea00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009fd0 tx=0x7ff2e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0014420 con 0x7ff2e8161210 2023-10-19T10:37:05.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e0014580 con 0x7ff2e8161210 2023-10-19T10:37:05.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e815aa70 con 0x7ff2e8161210 2023-10-19T10:37:05.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0014930 con 0x7ff2e8161210 2023-10-19T10:37:05.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.574+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e81f2350 con 0x7ff2e8161210 2023-10-19T10:37:05.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.575+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e815b170 con 0x7ff2e8161210 2023-10-19T10:37:05.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.575+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff2e0014a90 con 0x7ff2e8161210 2023-10-19T10:37:05.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.576+0000 7ff2cffff640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.576+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff2e0089240 con 0x7ff2e8161210 2023-10-19T10:37:05.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.576+0000 7ff2e77fe640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.577+0000 7ff2e77fe640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8007fd0 tx=0x7ff2d8007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.580+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff2e0057fb0 con 0x7ff2e8161210 2023-10-19T10:37:05.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.757+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff2e815af90 con 0x7ff2e8161210 2023-10-19T10:37:05.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.758+0000 7ff2cffff640 1 -- 172.21.15.50:0/2343432022 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff2e815af90 con 0x7ff2e8161210 2023-10-19T10:37:05.761 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:05.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 msgr2=0x7ff2d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8007fd0 tx=0x7ff2d8007e80 comp rx=0 tx=0).stop 2023-10-19T10:37:05.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 msgr2=0x7ff2e80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 0x7ff2e80cea00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009fd0 tx=0x7ff2e000e4b0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 shutdown_connections 2023-10-19T10:37:05.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e8161210 0x7ff2e80cea00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e80b38f0 0x7ff2e80ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 --2- 172.21.15.50:0/2343432022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e80b3090 0x7ff2e8157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 >> 172.21.15.50:0/2343432022 conn(0x7ff2e809b960 msgr2=0x7ff2e80b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:05.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 shutdown_connections 2023-10-19T10:37:05.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.760+0000 7ff2edf27640 1 -- 172.21.15.50:0/2343432022 wait complete. 2023-10-19T10:37:05.773 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:05.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.879+0000 7fcf3a9a7640 1 Processor -- start 2023-10-19T10:37:05.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.879+0000 7fcf3a9a7640 1 -- start start 2023-10-19T10:37:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf3a9a7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcf34069a70 0x7fcf340c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf3a9a7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf340cb6f0 0x7fcf3406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf3a9a7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf340ca120 con 0x7fcf34069e50 2023-10-19T10:37:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf3a9a7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf34071b90 con 0x7fcf34069a70 2023-10-19T10:37:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf3a9a7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf34071310 con 0x7fcf340cb6f0 2023-10-19T10:37:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf33fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcf34069a70 0x7fcf340c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf38f44640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fcf34069e50 0x7fcf3406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57094/0 (socket says 172.21.15.50:57094) 2023-10-19T10:37:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.880+0000 7fcf38f44640 1 -- 172.21.15.50:0/2904769706 learned_addr learned my addr 172.21.15.50:0/2904769706 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf337fe640 1 --2- 172.21.15.50:0/2904769706 >> v2:172.21.15.183:3300/0 conn(0x7fcf340cb6f0 0x7fcf3406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf33fff640 1 -- 172.21.15.50:0/2904769706 >> v2:172.21.15.183:3300/0 conn(0x7fcf340cb6f0 msgr2=0x7fcf3406dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf33fff640 1 --2- 172.21.15.50:0/2904769706 >> v2:172.21.15.183:3300/0 conn(0x7fcf340cb6f0 0x7fcf3406dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf33fff640 1 -- 172.21.15.50:0/2904769706 >> v1:172.21.15.50:6789/0 conn(0x7fcf34069e50 legacy=0x7fcf3406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf33fff640 1 -- 172.21.15.50:0/2904769706 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340715d0 con 0x7fcf34069a70 2023-10-19T10:37:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf32ffd640 1 -- 172.21.15.50:0/2904769706 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867923779 0 0) 0x7fcf340ca120 con 0x7fcf34069e50 2023-10-19T10:37:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf33fff640 1 --2- 172.21.15.50:0/2904769706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf340c97d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406f300 tx=0x7fcf2402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aea1742b192700a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.881+0000 7fcf32ffd640 1 -- 172.21.15.50:0/2904769706 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24031710 con 0x7fcf34069a70 2023-10-19T10:37:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf32ffd640 1 -- 172.21.15.50:0/2904769706 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2402f050 con 0x7fcf34069a70 2023-10-19T10:37:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf32ffd640 1 -- 172.21.15.50:0/2904769706 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24031a90 con 0x7fcf34069a70 2023-10-19T10:37:05.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2904769706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 msgr2=0x7fcf340c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2904769706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf340c97d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406f300 tx=0x7fcf2402b7b0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2904769706 shutdown_connections 2023-10-19T10:37:05.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2904769706 >> v2:172.21.15.183:3300/0 conn(0x7fcf340cb6f0 0x7fcf3406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2904769706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf340c97d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2904769706 >> 172.21.15.50:0/2904769706 conn(0x7fcf340cbc00 msgr2=0x7fcf340cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:05.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2904769706 shutdown_connections 2023-10-19T10:37:05.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.882+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2904769706 wait complete. 2023-10-19T10:37:05.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.883+0000 7fcf3a9a7640 1 Processor -- start 2023-10-19T10:37:05.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.883+0000 7fcf3a9a7640 1 -- start start 2023-10-19T10:37:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf3a9a7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf34153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf3a9a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf34069e50 0x7fcf3415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf3a9a7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf340cb6f0 0x7fcf341568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf3a9a7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34154450 con 0x7fcf34069a70 2023-10-19T10:37:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf3a9a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34143690 con 0x7fcf34069e50 2023-10-19T10:37:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf3a9a7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34154b70 con 0x7fcf340cb6f0 2023-10-19T10:37:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf34153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf34153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39760/0 (socket says 172.21.15.50:39760) 2023-10-19T10:37:05.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.884+0000 7fcf33fff640 1 -- 172.21.15.50:0/2251526495 learned_addr learned my addr 172.21.15.50:0/2251526495 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf337fe640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf34069e50 0x7fcf3415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf38f44640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf340cb6f0 0x7fcf341568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf33fff640 1 -- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf340cb6f0 msgr2=0x7fcf341568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf33fff640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf340cb6f0 0x7fcf341568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf33fff640 1 -- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf34069e50 msgr2=0x7fcf3415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf33fff640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf34069e50 0x7fcf3415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf33fff640 1 -- 172.21.15.50:0/2251526495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24009400 con 0x7fcf34069a70 2023-10-19T10:37:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf337fe640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf34069e50 0x7fcf3415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf38f44640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf340cb6f0 0x7fcf341568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:05.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf33fff640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf34153f10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf24000c00 tx=0x7fcf240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24004230 con 0x7fcf34069a70 2023-10-19T10:37:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf240043b0 con 0x7fcf34069a70 2023-10-19T10:37:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf34143040 con 0x7fcf34069a70 2023-10-19T10:37:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24004520 con 0x7fcf34069a70 2023-10-19T10:37:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.885+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf34145d70 con 0x7fcf34069a70 2023-10-19T10:37:05.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.886+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3404e420 con 0x7fcf34069a70 2023-10-19T10:37:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.887+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf2403a050 con 0x7fcf34069a70 2023-10-19T10:37:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.887+0000 7fcf30ff9640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf2006a050 0x7fcf2006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.887+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcf240aa330 con 0x7fcf34069a70 2023-10-19T10:37:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.887+0000 7fcf337fe640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf2006a050 0x7fcf2006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:05.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.888+0000 7fcf337fe640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf2006a050 0x7fcf2006c510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf340b4250 tx=0x7fcf28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:05.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:05.891+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fcf24075850 con 0x7fcf34069a70 2023-10-19T10:37:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.063+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf3415ded0 con 0x7fcf34069a70 2023-10-19T10:37:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.064+0000 7fcf30ff9640 1 -- 172.21.15.50:0/2251526495 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf24078e70 con 0x7fcf34069a70 2023-10-19T10:37:06.067 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf2006a050 msgr2=0x7fcf2006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf2006a050 0x7fcf2006c510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf340b4250 tx=0x7fcf28008040 comp rx=0 tx=0).stop 2023-10-19T10:37:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 msgr2=0x7fcf34153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf34153f10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf24000c00 tx=0x7fcf240046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 shutdown_connections 2023-10-19T10:37:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf2006a050 0x7fcf2006c510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf340cb6f0 0x7fcf341568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf34069e50 0x7fcf3415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 --2- 172.21.15.50:0/2251526495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf34069a70 0x7fcf34153f10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 >> 172.21.15.50:0/2251526495 conn(0x7fcf340cbc00 msgr2=0x7fcf3414fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 shutdown_connections 2023-10-19T10:37:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.066+0000 7fcf3a9a7640 1 -- 172.21.15.50:0/2251526495 wait complete. 2023-10-19T10:37:06.079 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:06.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.186+0000 7f3d2948b640 1 Processor -- start 2023-10-19T10:37:06.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.186+0000 7f3d2948b640 1 -- start start 2023-10-19T10:37:06.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d2948b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b38f0 0x7f3d240d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d2948b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3d240b3090 0x7f3d24160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d2948b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3d240b3cd0 0x7f3d24161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d2948b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d240b0190 con 0x7f3d240b3090 2023-10-19T10:37:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d2948b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d240af250 con 0x7f3d240b3cd0 2023-10-19T10:37:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d2948b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240af510 con 0x7f3d240b38f0 2023-10-19T10:37:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3d240b3090 0x7f3d24160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3d240b3090 0x7f3d24160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39764/0 (socket says 172.21.15.50:39764) 2023-10-19T10:37:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 -- 172.21.15.50:0/2161694830 learned_addr learned my addr 172.21.15.50:0/2161694830 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d237fe640 1 --2- 172.21.15.50:0/2161694830 >> v2:172.21.15.183:3300/0 conn(0x7f3d240b3cd0 0x7f3d24161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 -- 172.21.15.50:0/2161694830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b38f0 msgr2=0x7f3d240d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d22ffd640 1 --2- 172.21.15.50:0/2161694830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b38f0 0x7f3d240d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 --2- 172.21.15.50:0/2161694830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b38f0 0x7f3d240d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 -- 172.21.15.50:0/2161694830 >> v2:172.21.15.183:3300/0 conn(0x7f3d240b3cd0 msgr2=0x7f3d24161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 --2- 172.21.15.50:0/2161694830 >> v2:172.21.15.183:3300/0 conn(0x7f3d240b3cd0 0x7f3d24161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d227fc640 1 -- 172.21.15.50:0/2161694830 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240af7f0 con 0x7f3d240b3090 2023-10-19T10:37:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.187+0000 7f3d237fe640 1 --2- 172.21.15.50:0/2161694830 >> v2:172.21.15.183:3300/0 conn(0x7f3d240b3cd0 0x7f3d24161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.188+0000 7f3d227fc640 1 --2- 172.21.15.50:0/2161694830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3090 0x7f3d24160b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d240b3600 tx=0x7f3d0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da2b13dd1bc6329 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.188+0000 7f3d21ffb640 1 -- 172.21.15.50:0/2161694830 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c031870 con 0x7f3d240b3090 2023-10-19T10:37:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.188+0000 7f3d21ffb640 1 -- 172.21.15.50:0/2161694830 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0319d0 con 0x7f3d240b3090 2023-10-19T10:37:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.188+0000 7f3d21ffb640 1 -- 172.21.15.50:0/2161694830 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c031d60 con 0x7f3d240b3090 2023-10-19T10:37:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 -- 172.21.15.50:0/2161694830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3090 msgr2=0x7f3d24160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 --2- 172.21.15.50:0/2161694830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3090 0x7f3d24160b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d240b3600 tx=0x7f3d0c02b910 comp rx=0 tx=0).stop 2023-10-19T10:37:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 -- 172.21.15.50:0/2161694830 shutdown_connections 2023-10-19T10:37:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 --2- 172.21.15.50:0/2161694830 >> v2:172.21.15.183:3300/0 conn(0x7f3d240b3cd0 0x7f3d24161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 --2- 172.21.15.50:0/2161694830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3090 0x7f3d24160b00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 --2- 172.21.15.50:0/2161694830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b38f0 0x7f3d240d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 -- 172.21.15.50:0/2161694830 >> 172.21.15.50:0/2161694830 conn(0x7f3d2409b960 msgr2=0x7f3d240d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 -- 172.21.15.50:0/2161694830 shutdown_connections 2023-10-19T10:37:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.189+0000 7f3d2948b640 1 -- 172.21.15.50:0/2161694830 wait complete. 2023-10-19T10:37:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.190+0000 7f3d2948b640 1 Processor -- start 2023-10-19T10:37:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.190+0000 7f3d2948b640 1 -- start start 2023-10-19T10:37:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d2948b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b3090 0x7f3d24146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d2948b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d240b38f0 0x7f3d24146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d2948b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 0x7f3d2415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d2948b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24147100 con 0x7f3d240b3cd0 2023-10-19T10:37:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d2948b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24147320 con 0x7f3d240b38f0 2023-10-19T10:37:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d2948b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24147600 con 0x7f3d240b3090 2023-10-19T10:37:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 0x7f3d2415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 0x7f3d2415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39780/0 (socket says 172.21.15.50:39780) 2023-10-19T10:37:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 -- 172.21.15.50:0/376685279 learned_addr learned my addr 172.21.15.50:0/376685279 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d227fc640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d240b38f0 0x7f3d24146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d22ffd640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b3090 0x7f3d24146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 -- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b3090 msgr2=0x7f3d24146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b3090 0x7f3d24146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 -- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d240b38f0 msgr2=0x7f3d24146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d240b38f0 0x7f3d24146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 -- 172.21.15.50:0/376685279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0094d0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d227fc640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d240b38f0 0x7f3d24146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.191+0000 7f3d237fe640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 0x7f3d2415b510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c7c0 tx=0x7f3d1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.192+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004230 con 0x7f3d240b3cd0 2023-10-19T10:37:06.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.192+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c004390 con 0x7f3d240b3cd0 2023-10-19T10:37:06.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.192+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d241478e0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.192+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c007860 con 0x7f3d240b3cd0 2023-10-19T10:37:06.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.192+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d24147ea0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.193+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2404e420 con 0x7f3d240b3cd0 2023-10-19T10:37:06.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.193+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3d1c0079c0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.194+0000 7f3d1b7fe640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d0406a050 0x7f3d0406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.194+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3d1c088b00 con 0x7f3d240b3cd0 2023-10-19T10:37:06.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.194+0000 7f3d22ffd640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d0406a050 0x7f3d0406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.194+0000 7f3d22ffd640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d0406a050 0x7f3d0406c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d14008ec0 tx=0x7f3d14008410 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.198+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3d1c053fc0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.373+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d240b86b0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.373+0000 7f3d1b7fe640 1 -- 172.21.15.50:0/376685279 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d1c0575e0 con 0x7f3d240b3cd0 2023-10-19T10:37:06.377 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:06.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d0406a050 msgr2=0x7f3d0406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d0406a050 0x7f3d0406c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d14008ec0 tx=0x7f3d14008410 comp rx=0 tx=0).stop 2023-10-19T10:37:06.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 msgr2=0x7f3d2415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 0x7f3d2415b510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c7c0 tx=0x7f3d1c00cc80 comp rx=0 tx=0).stop 2023-10-19T10:37:06.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 shutdown_connections 2023-10-19T10:37:06.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3d0406a050 0x7f3d0406c510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d240b3cd0 0x7f3d2415b510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d240b38f0 0x7f3d24146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 --2- 172.21.15.50:0/376685279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d240b3090 0x7f3d24146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 >> 172.21.15.50:0/376685279 conn(0x7f3d2409b960 msgr2=0x7f3d240ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:06.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 shutdown_connections 2023-10-19T10:37:06.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.375+0000 7f3d2948b640 1 -- 172.21.15.50:0/376685279 wait complete. 2023-10-19T10:37:06.388 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:06.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.495+0000 7f6b4e560640 1 Processor -- start 2023-10-19T10:37:06.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.495+0000 7f6b4e560640 1 -- start start 2023-10-19T10:37:06.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4e560640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b480b4070 0x7f6b480b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4e560640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 0x7f6b480cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4e560640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b480d3480 con 0x7f6b4815c960 2023-10-19T10:37:06.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4e560640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b480b4990 con 0x7f6b480b4070 2023-10-19T10:37:06.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4e560640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4815d280 con 0x7f6b4815cdf0 2023-10-19T10:37:06.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4cafd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 0x7f6b480cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b47fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b480b4070 0x7f6b480b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4cafd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 0x7f6b480cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:53766/0 (socket says 172.21.15.50:53766) 2023-10-19T10:37:06.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.496+0000 7f6b4cafd640 1 -- 172.21.15.50:0/320406473 learned_addr learned my addr 172.21.15.50:0/320406473 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:06.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.497+0000 7f6b4cafd640 1 -- 172.21.15.50:0/320406473 >> v2:172.21.15.183:3300/0 conn(0x7f6b480b4070 msgr2=0x7f6b480b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:06.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.497+0000 7f6b4cafd640 1 --2- 172.21.15.50:0/320406473 >> v2:172.21.15.183:3300/0 conn(0x7f6b480b4070 0x7f6b480b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.497+0000 7f6b4cafd640 1 -- 172.21.15.50:0/320406473 >> v1:172.21.15.183:6789/0 conn(0x7f6b4815c960 legacy=0x7f6b480d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.497+0000 7f6b4cafd640 1 -- 172.21.15.50:0/320406473 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4815d560 con 0x7f6b4815cdf0 2023-10-19T10:37:06.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.497+0000 7f6b46ffd640 1 -- 172.21.15.50:0/320406473 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518635 0 0) 0x7f6b480d3480 con 0x7f6b4815c960 2023-10-19T10:37:06.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.497+0000 7f6b47fff640 1 --2- 172.21.15.50:0/320406473 >> v2:172.21.15.183:3300/0 conn(0x7f6b480b4070 0x7f6b480b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:06.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4cafd640 1 --2- 172.21.15.50:0/320406473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 0x7f6b480cd4b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009e50 tx=0x7f6b4002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfca5d483e3e8680 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b46ffd640 1 -- 172.21.15.50:0/320406473 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4002cb00 con 0x7f6b4815cdf0 2023-10-19T10:37:06.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b46ffd640 1 -- 172.21.15.50:0/320406473 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4002cc60 con 0x7f6b4815cdf0 2023-10-19T10:37:06.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b46ffd640 1 -- 172.21.15.50:0/320406473 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400343f0 con 0x7f6b4815cdf0 2023-10-19T10:37:06.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 -- 172.21.15.50:0/320406473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 msgr2=0x7f6b480cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 --2- 172.21.15.50:0/320406473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 0x7f6b480cd4b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009e50 tx=0x7f6b4002bf60 comp rx=0 tx=0).stop 2023-10-19T10:37:06.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 -- 172.21.15.50:0/320406473 shutdown_connections 2023-10-19T10:37:06.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 --2- 172.21.15.50:0/320406473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815cdf0 0x7f6b480cd4b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 --2- 172.21.15.50:0/320406473 >> v2:172.21.15.183:3300/0 conn(0x7f6b480b4070 0x7f6b480b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 -- 172.21.15.50:0/320406473 >> 172.21.15.50:0/320406473 conn(0x7f6b4809b970 msgr2=0x7f6b480d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:06.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 -- 172.21.15.50:0/320406473 shutdown_connections 2023-10-19T10:37:06.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.498+0000 7f6b4e560640 1 -- 172.21.15.50:0/320406473 wait complete. 2023-10-19T10:37:06.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.499+0000 7f6b4e560640 1 Processor -- start 2023-10-19T10:37:06.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 -- start start 2023-10-19T10:37:06.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 0x7f6b48143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815c960 0x7f6b480c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b48154f00 0x7f6b48142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48157960 con 0x7f6b480b4070 2023-10-19T10:37:06.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48157d80 con 0x7f6b48154f00 2023-10-19T10:37:06.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.500+0000 7f6b4e560640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480c9810 con 0x7f6b4815c960 2023-10-19T10:37:06.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 0x7f6b48143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 0x7f6b48143450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39792/0 (socket says 172.21.15.50:39792) 2023-10-19T10:37:06.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 -- 172.21.15.50:0/948479159 learned_addr learned my addr 172.21.15.50:0/948479159 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:06.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b4cafd640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b48154f00 0x7f6b48142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b477fe640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815c960 0x7f6b480c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 -- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815c960 msgr2=0x7f6b480c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815c960 0x7f6b480c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 -- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b48154f00 msgr2=0x7f6b48142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b48154f00 0x7f6b48142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 -- 172.21.15.50:0/948479159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0093a0 con 0x7f6b480b4070 2023-10-19T10:37:06.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b47fff640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 0x7f6b48143450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00ec10 tx=0x7f6b3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c009750 con 0x7f6b480b4070 2023-10-19T10:37:06.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c00cc30 con 0x7f6b480b4070 2023-10-19T10:37:06.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b40009b30 con 0x7f6b480b4070 2023-10-19T10:37:06.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c005990 con 0x7f6b480b4070 2023-10-19T10:37:06.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.501+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b48157580 con 0x7f6b480b4070 2023-10-19T10:37:06.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.502+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b48157740 con 0x7f6b480b4070 2023-10-19T10:37:06.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.503+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6b3c005af0 con 0x7f6b480b4070 2023-10-19T10:37:06.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.503+0000 7f6b44ff9640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b3406a080 0x7f6b3406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.503+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6b3c088cc0 con 0x7f6b480b4070 2023-10-19T10:37:06.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.503+0000 7f6b477fe640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b3406a080 0x7f6b3406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.504+0000 7f6b477fe640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b3406a080 0x7f6b3406c540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b480cc630 tx=0x7f6b38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.508+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6b3c054170 con 0x7f6b480b4070 2023-10-19T10:37:06.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.680+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6b4804e420 con 0x7f6b480b4070 2023-10-19T10:37:06.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.681+0000 7f6b44ff9640 1 -- 172.21.15.50:0/948479159 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6b3c0200a0 con 0x7f6b480b4070 2023-10-19T10:37:06.684 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:06.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b3406a080 msgr2=0x7f6b3406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b3406a080 0x7f6b3406c540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b480cc630 tx=0x7f6b38009040 comp rx=0 tx=0).stop 2023-10-19T10:37:06.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 msgr2=0x7f6b48143450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 0x7f6b48143450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00ec10 tx=0x7f6b3c00c520 comp rx=0 tx=0).stop 2023-10-19T10:37:06.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 shutdown_connections 2023-10-19T10:37:06.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b3406a080 0x7f6b3406c540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b48154f00 0x7f6b48142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b4815c960 0x7f6b480c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 --2- 172.21.15.50:0/948479159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b480b4070 0x7f6b48143450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 >> 172.21.15.50:0/948479159 conn(0x7f6b4809b970 msgr2=0x7f6b4806ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:06.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 shutdown_connections 2023-10-19T10:37:06.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.683+0000 7f6b4e560640 1 -- 172.21.15.50:0/948479159 wait complete. 2023-10-19T10:37:06.696 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:37:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.801+0000 7f6564b02640 1 Processor -- start 2023-10-19T10:37:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.801+0000 7f6564b02640 1 -- start start 2023-10-19T10:37:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.801+0000 7f6564b02640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f65600b2400 0x7f656015d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.801+0000 7f6564b02640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65601610a0 con 0x7f65600ce3c0 2023-10-19T10:37:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.801+0000 7f6564b02640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65600ce7a0 con 0x7f6560160100 2023-10-19T10:37:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.801+0000 7f6564b02640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656015cad0 con 0x7f65600b2400 2023-10-19T10:37:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f6557fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f65600b2400 0x7f656015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f655e59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f65600ce3c0 0x7f65600b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38346/0 (socket says 172.21.15.50:38346) 2023-10-19T10:37:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f6557fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f65600b2400 0x7f656015d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38412/0 (socket says 172.21.15.50:38412) 2023-10-19T10:37:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f655e59c640 1 -- 172.21.15.50:0/4261322303 learned_addr learned my addr 172.21.15.50:0/4261322303 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f6557fff640 1 -- 172.21.15.50:0/4261322303 >> v1:172.21.15.183:6789/0 conn(0x7f6560160100 legacy=0x7f6560160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f6557fff640 1 -- 172.21.15.50:0/4261322303 >> v1:172.21.15.50:6789/0 conn(0x7f65600ce3c0 legacy=0x7f65600b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f6557fff640 1 -- 172.21.15.50:0/4261322303 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560160dc0 con 0x7f65600b2400 2023-10-19T10:37:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.802+0000 7f655dd9b640 1 -- 172.21.15.50:0/4261322303 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306499585 0 0) 0x7f65601610a0 con 0x7f65600ce3c0 2023-10-19T10:37:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6557fff640 1 --2- 172.21.15.50:0/4261322303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600b2400 0x7f656015d790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65480097a0 tx=0x7f654802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c3399fd2edea56 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f655dd9b640 1 -- 172.21.15.50:0/4261322303 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548030610 con 0x7f65600b2400 2023-10-19T10:37:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f655dd9b640 1 -- 172.21.15.50:0/4261322303 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548030770 con 0x7f65600b2400 2023-10-19T10:37:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f655dd9b640 1 -- 172.21.15.50:0/4261322303 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548030b00 con 0x7f65600b2400 2023-10-19T10:37:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 -- 172.21.15.50:0/4261322303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600b2400 msgr2=0x7f656015d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 --2- 172.21.15.50:0/4261322303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600b2400 0x7f656015d790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65480097a0 tx=0x7f654802b960 comp rx=0 tx=0).stop 2023-10-19T10:37:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 -- 172.21.15.50:0/4261322303 shutdown_connections 2023-10-19T10:37:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 --2- 172.21.15.50:0/4261322303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600b2400 0x7f656015d790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 -- 172.21.15.50:0/4261322303 >> 172.21.15.50:0/4261322303 conn(0x7f656006f3f0 msgr2=0x7f656006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 -- 172.21.15.50:0/4261322303 shutdown_connections 2023-10-19T10:37:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.803+0000 7f6564b02640 1 -- 172.21.15.50:0/4261322303 wait complete. 2023-10-19T10:37:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 Processor -- start 2023-10-19T10:37:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 -- start start 2023-10-19T10:37:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65600b2400 0x7f65600b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 0x7f656015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6560160100 0x7f656015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65601f2cf0 con 0x7f65600ce3c0 2023-10-19T10:37:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600b5e50 con 0x7f65600b2400 2023-10-19T10:37:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.805+0000 7f6564b02640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65601f2070 con 0x7f6560160100 2023-10-19T10:37:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 0x7f656015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 0x7f656015e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38424/0 (socket says 172.21.15.50:38424) 2023-10-19T10:37:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 -- 172.21.15.50:0/909539780 learned_addr learned my addr 172.21.15.50:0/909539780 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f655e59c640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65600b2400 0x7f65600b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f655ed9d640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6560160100 0x7f656015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 -- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6560160100 msgr2=0x7f656015ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6560160100 0x7f656015ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 -- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65600b2400 msgr2=0x7f65600b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65600b2400 0x7f65600b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 -- 172.21.15.50:0/909539780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6548009400 con 0x7f65600ce3c0 2023-10-19T10:37:06.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6557fff640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 0x7f656015e7e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65480098c0 tx=0x7f6548004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654803e040 con 0x7f65600ce3c0 2023-10-19T10:37:06.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548039cb0 con 0x7f65600ce3c0 2023-10-19T10:37:06.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65601f2350 con 0x7f65600ce3c0 2023-10-19T10:37:06.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548044630 con 0x7f65600ce3c0 2023-10-19T10:37:06.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.806+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65601f2910 con 0x7f65600ce3c0 2023-10-19T10:37:06.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.807+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6560073990 con 0x7f65600ce3c0 2023-10-19T10:37:06.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.808+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f654804d3b0 con 0x7f65600ce3c0 2023-10-19T10:37:06.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.808+0000 7f6556ffd640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f654006a050 0x7f654006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:06.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.808+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65480aad90 con 0x7f65600ce3c0 2023-10-19T10:37:06.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.809+0000 7f655e59c640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f654006a050 0x7f654006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:06.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.809+0000 7f655e59c640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f654006a050 0x7f654006c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6550007f70 tx=0x7f6550007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:06.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:06.813+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f65480762d0 con 0x7f65600ce3c0 2023-10-19T10:37:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.000+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f656015fe20 con 0x7f65600ce3c0 2023-10-19T10:37:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.000+0000 7f6556ffd640 1 -- 172.21.15.50:0/909539780 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f65480798f0 con 0x7f65600ce3c0 2023-10-19T10:37:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:37:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f654006a050 msgr2=0x7f654006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f654006a050 0x7f654006c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6550007f70 tx=0x7f6550007f00 comp rx=0 tx=0).stop 2023-10-19T10:37:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 msgr2=0x7f656015e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 0x7f656015e7e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65480098c0 tx=0x7f6548004730 comp rx=0 tx=0).stop 2023-10-19T10:37:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 shutdown_connections 2023-10-19T10:37:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f654006a050 0x7f654006c510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6560160100 0x7f656015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65600ce3c0 0x7f656015e7e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 --2- 172.21.15.50:0/909539780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65600b2400 0x7f65600b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 >> 172.21.15.50:0/909539780 conn(0x7f656006f3f0 msgr2=0x7f6560072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 shutdown_connections 2023-10-19T10:37:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.002+0000 7f6564b02640 1 -- 172.21.15.50:0/909539780 wait complete. 2023-10-19T10:37:07.015 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:37:07.016 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:07.031 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:07.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.175+0000 7f84e9911640 1 Processor -- start 2023-10-19T10:37:07.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.175+0000 7f84e9911640 1 -- start start 2023-10-19T10:37:07.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e9911640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 0x7f84e406b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e9911640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40ca480 con 0x7f84e40cd7d0 2023-10-19T10:37:07.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e9911640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e4159990 con 0x7f84e4159280 2023-10-19T10:37:07.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e9911640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e415d500 con 0x7f84e415cd90 2023-10-19T10:37:07.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84dbfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f84e40cd7d0 0x7f84e4159e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38348/0 (socket says 172.21.15.50:38348) 2023-10-19T10:37:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84dbfff640 1 -- 172.21.15.50:0/207766496 learned_addr learned my addr 172.21.15.50:0/207766496 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e37fe640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 0x7f84e406b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127051871 0 0) 0x7f84e40ca480 con 0x7f84e40cd7d0 2023-10-19T10:37:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d0003520 con 0x7f84e40cd7d0 2023-10-19T10:37:07.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 582201710 0 0) 0x7f84d0003520 con 0x7f84e40cd7d0 2023-10-19T10:37:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 msgr2=0x7f84e406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 0x7f84e406b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 >> v1:172.21.15.183:6789/0 conn(0x7f84e4159280 legacy=0x7f84e40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e415d7e0 con 0x7f84e40cd7d0 2023-10-19T10:37:07.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f84cc001670 con 0x7f84e40cd7d0 2023-10-19T10:37:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 >> v1:172.21.15.50:6789/0 conn(0x7f84e40cd7d0 legacy=0x7f84e4159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 0x7f84d00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84d0003f20 0x7f84d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0004d00 0x7f84d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e415d500 con 0x7f84d0003f20 2023-10-19T10:37:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d000ba60 con 0x7f84d0004d00 2023-10-19T10:37:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d000bc20 con 0x7f84e415cd90 2023-10-19T10:37:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.176+0000 7f84dbfff640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84d0003f20 0x7f84d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84e2ffd640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 0x7f84d00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84e37fe640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0004d00 0x7f84d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84dbfff640 1 -- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 msgr2=0x7f84d00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:07.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84dbfff640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e415cd90 0x7f84d00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84dbfff640 1 -- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0004d00 msgr2=0x7f84d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:07.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84dbfff640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0004d00 0x7f84d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.177+0000 7f84dbfff640 1 -- 172.21.15.50:0/207766496 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0003520 con 0x7f84d0003f20 2023-10-19T10:37:07.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84dbfff640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84d0003f20 0x7f84d00047c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00c240 tx=0x7f84d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91687a547f327b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc005ad0 con 0x7f84d0003f20 2023-10-19T10:37:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc005c50 con 0x7f84d0003f20 2023-10-19T10:37:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e27fc640 1 -- 172.21.15.50:0/207766496 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc02c8b0 con 0x7f84d0003f20 2023-10-19T10:37:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e9911640 1 -- 172.21.15.50:0/207766496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84d0003f20 msgr2=0x7f84d00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e9911640 1 --2- 172.21.15.50:0/207766496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84d0003f20 0x7f84d00047c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00c240 tx=0x7f84d00057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e37fe640 1 -- 172.21.15.50:0/207766496 reap_dead start 2023-10-19T10:37:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e9911640 1 -- 172.21.15.50:0/207766496 shutdown_connections 2023-10-19T10:37:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e9911640 1 -- 172.21.15.50:0/207766496 >> 172.21.15.50:0/207766496 conn(0x7f84e4069e90 msgr2=0x7f84e40afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e9911640 1 -- 172.21.15.50:0/207766496 shutdown_connections 2023-10-19T10:37:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.178+0000 7f84e9911640 1 -- 172.21.15.50:0/207766496 wait complete. 2023-10-19T10:37:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.179+0000 7f84e9911640 1 Processor -- start 2023-10-19T10:37:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.179+0000 7f84e9911640 1 -- start start 2023-10-19T10:37:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e9911640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0003f20 0x7f84e41f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e9911640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 0x7f84e40ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e9911640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e40c1b40 0x7f84e40c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e9911640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40c2500 con 0x7f84e40abfd0 2023-10-19T10:37:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e9911640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40ac970 con 0x7f84d0003f20 2023-10-19T10:37:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e9911640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41f2710 con 0x7f84e40c1b40 2023-10-19T10:37:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 0x7f84e40ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 0x7f84e40ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38428/0 (socket says 172.21.15.50:38428) 2023-10-19T10:37:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84dbfff640 1 -- 172.21.15.50:0/2853026300 learned_addr learned my addr 172.21.15.50:0/2853026300 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e2ffd640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0003f20 0x7f84e41f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.180+0000 7f84e37fe640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e40c1b40 0x7f84e40c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84dbfff640 1 -- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e40c1b40 msgr2=0x7f84e40c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84dbfff640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e40c1b40 0x7f84e40c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84dbfff640 1 -- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0003f20 msgr2=0x7f84e41f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84dbfff640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0003f20 0x7f84e41f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84dbfff640 1 -- 172.21.15.50:0/2853026300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4148ee0 con 0x7f84e40abfd0 2023-10-19T10:37:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84e37fe640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e40c1b40 0x7f84e40c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84dbfff640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 0x7f84e40ac430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02c450 tx=0x7f84cc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc039070 con 0x7f84e40abfd0 2023-10-19T10:37:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc02ccc0 con 0x7f84e40abfd0 2023-10-19T10:37:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e40acb90 con 0x7f84e40abfd0 2023-10-19T10:37:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc03c040 con 0x7f84e40abfd0 2023-10-19T10:37:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.181+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e40ad150 con 0x7f84e40abfd0 2023-10-19T10:37:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.182+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e40c19b0 con 0x7f84e40abfd0 2023-10-19T10:37:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.182+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f84cc032070 con 0x7f84e40abfd0 2023-10-19T10:37:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.183+0000 7f84e0ff9640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f84c406a050 0x7f84c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.183+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f84cc0a9be0 con 0x7f84e40abfd0 2023-10-19T10:37:07.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.183+0000 7f84e2ffd640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f84c406a050 0x7f84c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.184+0000 7f84e2ffd640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f84c406a050 0x7f84c406c510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d40017e0 tx=0x7f84d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.187+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f84cc074fe0 con 0x7f84e40abfd0 2023-10-19T10:37:07.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.368+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84e4073990 con 0x7f84e40abfd0 2023-10-19T10:37:07.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.369+0000 7f84e0ff9640 1 -- 172.21.15.50:0/2853026300 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f84cc078600 con 0x7f84e40abfd0 2023-10-19T10:37:07.372 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:07.372 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:07.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f84c406a050 msgr2=0x7f84c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f84c406a050 0x7f84c406c510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d40017e0 tx=0x7f84d4009040 comp rx=0 tx=0).stop 2023-10-19T10:37:07.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 msgr2=0x7f84e40ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 0x7f84e40ac430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02c450 tx=0x7f84cc001a00 comp rx=0 tx=0).stop 2023-10-19T10:37:07.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 shutdown_connections 2023-10-19T10:37:07.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f84c406a050 0x7f84c406c510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e40c1b40 0x7f84e40c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84e40abfd0 0x7f84e40ac430 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 --2- 172.21.15.50:0/2853026300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84d0003f20 0x7f84e41f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 >> 172.21.15.50:0/2853026300 conn(0x7f84e4069e90 msgr2=0x7f84e40adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 shutdown_connections 2023-10-19T10:37:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.371+0000 7f84e9911640 1 -- 172.21.15.50:0/2853026300 wait complete. 2023-10-19T10:37:07.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:07.385 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:07.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.491+0000 7f452b5f2640 1 Processor -- start 2023-10-19T10:37:07.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.492+0000 7f452b5f2640 1 -- start start 2023-10-19T10:37:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.492+0000 7f452b5f2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f45240b4070 0x7f45240b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.492+0000 7f452b5f2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f452415cdc0 0x7f45240cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.492+0000 7f452b5f2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45240d3480 con 0x7f452415c960 2023-10-19T10:37:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.492+0000 7f452b5f2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45240b4990 con 0x7f452415cdc0 2023-10-19T10:37:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.492+0000 7f452b5f2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452415d280 con 0x7f45240b4070 2023-10-19T10:37:07.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f452415cdc0 0x7f45240cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f452415cdc0 0x7f45240cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38442/0 (socket says 172.21.15.50:38442) 2023-10-19T10:37:07.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 -- 172.21.15.50:0/817803732 learned_addr learned my addr 172.21.15.50:0/817803732 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:07.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f452938e640 1 --2- 172.21.15.50:0/817803732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f45240b4070 0x7f45240b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 -- 172.21.15.50:0/817803732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f45240b4070 msgr2=0x7f45240b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:07.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 --2- 172.21.15.50:0/817803732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f45240b4070 0x7f45240b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 -- 172.21.15.50:0/817803732 >> v1:172.21.15.183:6789/0 conn(0x7f452415c960 legacy=0x7f45240d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.493+0000 7f4529b8f640 1 -- 172.21.15.50:0/817803732 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452415d560 con 0x7f452415cdc0 2023-10-19T10:37:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f4529b8f640 1 --2- 172.21.15.50:0/817803732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f452415cdc0 0x7f45240cd4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f451c009e50 tx=0x7f451c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d6f665383cfc40 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f451bfff640 1 -- 172.21.15.50:0/817803732 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c02cb00 con 0x7f452415cdc0 2023-10-19T10:37:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f451bfff640 1 -- 172.21.15.50:0/817803732 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c02cc60 con 0x7f452415cdc0 2023-10-19T10:37:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f451bfff640 1 -- 172.21.15.50:0/817803732 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0343f0 con 0x7f452415cdc0 2023-10-19T10:37:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 -- 172.21.15.50:0/817803732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f452415cdc0 msgr2=0x7f45240cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 --2- 172.21.15.50:0/817803732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f452415cdc0 0x7f45240cd4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f451c009e50 tx=0x7f451c02bf60 comp rx=0 tx=0).stop 2023-10-19T10:37:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 -- 172.21.15.50:0/817803732 shutdown_connections 2023-10-19T10:37:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 --2- 172.21.15.50:0/817803732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f452415cdc0 0x7f45240cd4b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 --2- 172.21.15.50:0/817803732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f45240b4070 0x7f45240b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 -- 172.21.15.50:0/817803732 >> 172.21.15.50:0/817803732 conn(0x7f452409b970 msgr2=0x7f45240d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 -- 172.21.15.50:0/817803732 shutdown_connections 2023-10-19T10:37:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.494+0000 7f452b5f2640 1 -- 172.21.15.50:0/817803732 wait complete. 2023-10-19T10:37:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.495+0000 7f452b5f2640 1 Processor -- start 2023-10-19T10:37:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 -- start start 2023-10-19T10:37:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 0x7f45240b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f452415c960 0x7f45240cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f452415cdc0 0x7f45240c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45240c8f40 con 0x7f45240b4070 2023-10-19T10:37:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45240b0710 con 0x7f452415cdc0 2023-10-19T10:37:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452b5f2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45240b09f0 con 0x7f452415c960 2023-10-19T10:37:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452938e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 0x7f45240b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452938e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 0x7f45240b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38446/0 (socket says 172.21.15.50:38446) 2023-10-19T10:37:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.496+0000 7f452938e640 1 -- 172.21.15.50:0/259680925 learned_addr learned my addr 172.21.15.50:0/259680925 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f4529b8f640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f452415cdc0 0x7f45240c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f4528b8d640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f452415c960 0x7f45240cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452938e640 1 -- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f452415c960 msgr2=0x7f45240cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452938e640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f452415c960 0x7f45240cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452938e640 1 -- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f452415cdc0 msgr2=0x7f45240c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452938e640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f452415cdc0 0x7f45240c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452938e640 1 -- 172.21.15.50:0/259680925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009b30 con 0x7f45240b4070 2023-10-19T10:37:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f4528b8d640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f452415c960 0x7f45240cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452938e640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 0x7f45240b17b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f452000e900 tx=0x7f452000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520009660 con 0x7f45240b4070 2023-10-19T10:37:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000ca00 con 0x7f45240b4070 2023-10-19T10:37:07.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000cd70 con 0x7f45240b4070 2023-10-19T10:37:07.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45240b0cd0 con 0x7f45240b4070 2023-10-19T10:37:07.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.497+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45240b1290 con 0x7f45240b4070 2023-10-19T10:37:07.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.498+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45240b1450 con 0x7f45240b4070 2023-10-19T10:37:07.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.499+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4520013470 con 0x7f45240b4070 2023-10-19T10:37:07.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.499+0000 7f4519ffb640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f450806a080 0x7f450806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.499+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4520088d20 con 0x7f45240b4070 2023-10-19T10:37:07.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.499+0000 7f4528b8d640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f450806a080 0x7f450806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.500+0000 7f4528b8d640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f450806a080 0x7f450806c540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4514000f60 tx=0x7f4514009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.503+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4520054260 con 0x7f45240b4070 2023-10-19T10:37:07.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.678+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45240ce950 con 0x7f45240b4070 2023-10-19T10:37:07.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.679+0000 7f4519ffb640 1 -- 172.21.15.50:0/259680925 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4520021020 con 0x7f45240b4070 2023-10-19T10:37:07.682 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:07.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f450806a080 msgr2=0x7f450806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f450806a080 0x7f450806c540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4514000f60 tx=0x7f4514009040 comp rx=0 tx=0).stop 2023-10-19T10:37:07.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 msgr2=0x7f45240b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 0x7f45240b17b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f452000e900 tx=0x7f452000edc0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 shutdown_connections 2023-10-19T10:37:07.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f450806a080 0x7f450806c540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f452415cdc0 0x7f45240c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f452415c960 0x7f45240cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 --2- 172.21.15.50:0/259680925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45240b4070 0x7f45240b17b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 >> 172.21.15.50:0/259680925 conn(0x7f452409b970 msgr2=0x7f452415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 shutdown_connections 2023-10-19T10:37:07.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.681+0000 7f452b5f2640 1 -- 172.21.15.50:0/259680925 wait complete. 2023-10-19T10:37:07.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:07.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 Processor -- start 2023-10-19T10:37:07.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 -- start start 2023-10-19T10:37:07.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09940b4070 0x7f09940b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f099415c960 0x7f09940d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f099415cdc0 0x7f0994160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09940b4990 con 0x7f099415cdc0 2023-10-19T10:37:07.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099415d280 con 0x7f099415c960 2023-10-19T10:37:07.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.801+0000 7f0999692640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099415d560 con 0x7f09940b4070 2023-10-19T10:37:07.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f099415cdc0 0x7f0994160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f099415cdc0 0x7f0994160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38458/0 (socket says 172.21.15.50:38458) 2023-10-19T10:37:07.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 -- 172.21.15.50:0/3766580565 learned_addr learned my addr 172.21.15.50:0/3766580565 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:07.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f098bfff640 1 --2- 172.21.15.50:0/3766580565 >> v2:172.21.15.183:3300/0 conn(0x7f099415c960 0x7f09940d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f0992ffd640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09940b4070 0x7f09940b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 -- 172.21.15.50:0/3766580565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09940b4070 msgr2=0x7f09940b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09940b4070 0x7f09940b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 -- 172.21.15.50:0/3766580565 >> v2:172.21.15.183:3300/0 conn(0x7f099415c960 msgr2=0x7f09940d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 --2- 172.21.15.50:0/3766580565 >> v2:172.21.15.183:3300/0 conn(0x7f099415c960 0x7f09940d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f09937fe640 1 -- 172.21.15.50:0/3766580565 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099415d840 con 0x7f099415cdc0 2023-10-19T10:37:07.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.802+0000 7f0992ffd640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09940b4070 0x7f09940b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:07.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f09937fe640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f099415cdc0 0x7f0994160280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f099408a430 tx=0x7f098c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bac747a5fadf90 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f09927fc640 1 -- 172.21.15.50:0/3766580565 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c02ca70 con 0x7f099415cdc0 2023-10-19T10:37:07.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f09927fc640 1 -- 172.21.15.50:0/3766580565 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c02cbd0 con 0x7f099415cdc0 2023-10-19T10:37:07.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f09927fc640 1 -- 172.21.15.50:0/3766580565 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0343f0 con 0x7f099415cdc0 2023-10-19T10:37:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 -- 172.21.15.50:0/3766580565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f099415cdc0 msgr2=0x7f0994160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f099415cdc0 0x7f0994160280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f099408a430 tx=0x7f098c02bed0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 -- 172.21.15.50:0/3766580565 shutdown_connections 2023-10-19T10:37:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f099415cdc0 0x7f0994160280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 --2- 172.21.15.50:0/3766580565 >> v2:172.21.15.183:3300/0 conn(0x7f099415c960 0x7f09940d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 --2- 172.21.15.50:0/3766580565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09940b4070 0x7f09940b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 -- 172.21.15.50:0/3766580565 >> 172.21.15.50:0/3766580565 conn(0x7f099409b970 msgr2=0x7f09940d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.803+0000 7f0999692640 1 -- 172.21.15.50:0/3766580565 shutdown_connections 2023-10-19T10:37:07.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.804+0000 7f0999692640 1 -- 172.21.15.50:0/3766580565 wait complete. 2023-10-19T10:37:07.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 Processor -- start 2023-10-19T10:37:07.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 -- start start 2023-10-19T10:37:07.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 0x7f09940b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f099415c960 0x7f09940b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099415cdc0 0x7f09940c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940c61e0 con 0x7f09940b4070 2023-10-19T10:37:07.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940c64c0 con 0x7f099415cdc0 2023-10-19T10:37:07.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0999692640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09940c67a0 con 0x7f099415c960 2023-10-19T10:37:07.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 0x7f09940b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 0x7f09940b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38468/0 (socket says 172.21.15.50:38468) 2023-10-19T10:37:07.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.805+0000 7f0992ffd640 1 -- 172.21.15.50:0/2841360164 learned_addr learned my addr 172.21.15.50:0/2841360164 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:07.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f09937fe640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099415cdc0 0x7f09940c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f098bfff640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f099415c960 0x7f09940b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0992ffd640 1 -- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f099415c960 msgr2=0x7f09940b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0992ffd640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f099415c960 0x7f09940b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0992ffd640 1 -- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099415cdc0 msgr2=0x7f09940c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0992ffd640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099415cdc0 0x7f09940c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0992ffd640 1 -- 172.21.15.50:0/2841360164 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009b30 con 0x7f09940b4070 2023-10-19T10:37:07.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f09937fe640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099415cdc0 0x7f09940c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:07.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0992ffd640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 0x7f09940b0630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f098400e810 tx=0x7f098400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984004030 con 0x7f09940b4070 2023-10-19T10:37:07.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0984004190 con 0x7f09940b4070 2023-10-19T10:37:07.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09940c6a80 con 0x7f09940b4070 2023-10-19T10:37:07.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984005990 con 0x7f09940b4070 2023-10-19T10:37:07.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.806+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09941514d0 con 0x7f09940b4070 2023-10-19T10:37:07.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.807+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099404e420 con 0x7f09940b4070 2023-10-19T10:37:07.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.808+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0984005af0 con 0x7f09940b4070 2023-10-19T10:37:07.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.808+0000 7f098b7fe640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f097406a050 0x7f097406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:07.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.808+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0984088930 con 0x7f09940b4070 2023-10-19T10:37:07.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.808+0000 7f098bfff640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f097406a050 0x7f097406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:07.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.809+0000 7f098bfff640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f097406a050 0x7f097406c510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09940b3fa0 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:07.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.812+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0984053df0 con 0x7f09940b4070 2023-10-19T10:37:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.989+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f099408c720 con 0x7f09940b4070 2023-10-19T10:37:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.989+0000 7f098b7fe640 1 -- 172.21.15.50:0/2841360164 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0984057410 con 0x7f09940b4070 2023-10-19T10:37:07.992 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f097406a050 msgr2=0x7f097406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f097406a050 0x7f097406c510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09940b3fa0 tx=0x7f097c008040 comp rx=0 tx=0).stop 2023-10-19T10:37:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 msgr2=0x7f09940b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 0x7f09940b0630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f098400e810 tx=0x7f098400ecd0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 shutdown_connections 2023-10-19T10:37:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f097406a050 0x7f097406c510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099415cdc0 0x7f09940c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f099415c960 0x7f09940b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 --2- 172.21.15.50:0/2841360164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09940b4070 0x7f09940b0630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 >> 172.21.15.50:0/2841360164 conn(0x7f099409b970 msgr2=0x7f099406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 shutdown_connections 2023-10-19T10:37:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:07.991+0000 7f0999692640 1 -- 172.21.15.50:0/2841360164 wait complete. 2023-10-19T10:37:08.004 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:08.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.111+0000 7f3880a78640 1 Processor -- start 2023-10-19T10:37:08.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.111+0000 7f3880a78640 1 -- start start 2023-10-19T10:37:08.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3880a78640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f387c15c6e0 0x7f387c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3880a78640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 0x7f387c0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3880a78640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c160190 con 0x7f387c0709c0 2023-10-19T10:37:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3880a78640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c1611a0 con 0x7f387c15c6e0 2023-10-19T10:37:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3880a78640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c15cac0 con 0x7f387c06af20 2023-10-19T10:37:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3879d9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f387c0709c0 0x7f387c06a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38354/0 (socket says 172.21.15.50:38354) 2023-10-19T10:37:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f3879d9b640 1 -- 172.21.15.50:0/4132346914 learned_addr learned my addr 172.21.15.50:0/4132346914 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387a59c640 1 --2- 172.21.15.50:0/4132346914 >> v2:172.21.15.183:3300/0 conn(0x7f387c15c6e0 0x7f387c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387ad9d640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 0x7f387c0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527041212 0 0) 0x7f387c160190 con 0x7f387c0709c0 2023-10-19T10:37:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3864003520 con 0x7f387c0709c0 2023-10-19T10:37:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104928755 0 0) 0x7f3864003520 con 0x7f387c0709c0 2023-10-19T10:37:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 msgr2=0x7f387c0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:08.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 0x7f387c0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 >> v2:172.21.15.183:3300/0 conn(0x7f387c15c6e0 msgr2=0x7f387c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:08.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 --2- 172.21.15.50:0/4132346914 >> v2:172.21.15.183:3300/0 conn(0x7f387c15c6e0 0x7f387c0ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c160d10 con 0x7f387c0709c0 2023-10-19T10:37:08.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f3870001670 con 0x7f387c0709c0 2023-10-19T10:37:08.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 >> v1:172.21.15.50:6789/0 conn(0x7f387c0709c0 legacy=0x7f387c06a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 0x7f3864003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3864003ef0 0x7f38640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3864004d00 0x7f3864005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1611a0 con 0x7f3864003ef0 2023-10-19T10:37:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386400b980 con 0x7f3864004d00 2023-10-19T10:37:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.112+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386400bac0 con 0x7f387c06af20 2023-10-19T10:37:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3864003ef0 0x7f38640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f387a59c640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 0x7f3864003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 -- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 msgr2=0x7f3864003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f387c06af20 0x7f3864003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 -- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3864004d00 msgr2=0x7f3864005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3864004d00 0x7f3864005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 -- 172.21.15.50:0/4132346914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864003520 con 0x7f3864003ef0 2023-10-19T10:37:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.113+0000 7f3879d9b640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3864003ef0 0x7f38640047c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f387000c290 tx=0x7f38640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42644e61051b95bd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870005ad0 con 0x7f3864003ef0 2023-10-19T10:37:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870005c30 con 0x7f3864003ef0 2023-10-19T10:37:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f387959a640 1 -- 172.21.15.50:0/4132346914 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387002c8c0 con 0x7f3864003ef0 2023-10-19T10:37:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f3880a78640 1 -- 172.21.15.50:0/4132346914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3864003ef0 msgr2=0x7f38640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f3880a78640 1 --2- 172.21.15.50:0/4132346914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3864003ef0 0x7f38640047c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f387000c290 tx=0x7f38640057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f387ad9d640 1 -- 172.21.15.50:0/4132346914 reap_dead start 2023-10-19T10:37:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f3880a78640 1 -- 172.21.15.50:0/4132346914 shutdown_connections 2023-10-19T10:37:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f3880a78640 1 -- 172.21.15.50:0/4132346914 >> 172.21.15.50:0/4132346914 conn(0x7f387c06fea0 msgr2=0x7f387c0702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f3880a78640 1 -- 172.21.15.50:0/4132346914 shutdown_connections 2023-10-19T10:37:08.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.114+0000 7f3880a78640 1 -- 172.21.15.50:0/4132346914 wait complete. 2023-10-19T10:37:08.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 Processor -- start 2023-10-19T10:37:08.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 -- start start 2023-10-19T10:37:08.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3864003ef0 0x7f387c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 0x7f387c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f387c06cfc0 0x7f387c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c06d980 con 0x7f387c14f480 2023-10-19T10:37:08.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0b3c00 con 0x7f387c06cfc0 2023-10-19T10:37:08.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3880a78640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0b2120 con 0x7f3864003ef0 2023-10-19T10:37:08.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.116+0000 7f3879d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 0x7f387c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 0x7f387c0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38496/0 (socket says 172.21.15.50:38496) 2023-10-19T10:37:08.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 -- 172.21.15.50:0/2855677992 learned_addr learned my addr 172.21.15.50:0/2855677992 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:08.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f387a59c640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3864003ef0 0x7f387c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f387ad9d640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f387c06cfc0 0x7f387c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 -- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3864003ef0 msgr2=0x7f387c0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3864003ef0 0x7f387c0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 -- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f387c06cfc0 msgr2=0x7f387c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f387c06cfc0 0x7f387c06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 -- 172.21.15.50:0/2855677992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0b2400 con 0x7f387c14f480 2023-10-19T10:37:08.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3879d9b640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 0x7f387c0b36c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f387002c4c0 tx=0x7f3870000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387003c040 con 0x7f387c14f480 2023-10-19T10:37:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870007290 con 0x7f387c14f480 2023-10-19T10:37:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c0b26e0 con 0x7f387c14f480 2023-10-19T10:37:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870007640 con 0x7f387c14f480 2023-10-19T10:37:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.117+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c0ae560 con 0x7f387c14f480 2023-10-19T10:37:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.118+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c0b2260 con 0x7f387c14f480 2023-10-19T10:37:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.119+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3870032070 con 0x7f387c14f480 2023-10-19T10:37:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.119+0000 7f386affd640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f385c06a050 0x7f385c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.119+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f38700a9bb0 con 0x7f387c14f480 2023-10-19T10:37:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.119+0000 7f387a59c640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f385c06a050 0x7f385c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.120+0000 7f387a59c640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f385c06a050 0x7f385c06c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f386c009b40 tx=0x7f386c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.123+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f387c0b2260 con 0x7f387c14f480 2023-10-19T10:37:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.298+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f387c0b2260 con 0x7f387c14f480 2023-10-19T10:37:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.299+0000 7f386affd640 1 -- 172.21.15.50:0/2855677992 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f387c0b2260 con 0x7f387c14f480 2023-10-19T10:37:08.302 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f385c06a050 msgr2=0x7f385c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f385c06a050 0x7f385c06c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f386c009b40 tx=0x7f386c009190 comp rx=0 tx=0).stop 2023-10-19T10:37:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 msgr2=0x7f387c0b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 0x7f387c0b36c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f387002c4c0 tx=0x7f3870000fe0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 shutdown_connections 2023-10-19T10:37:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f385c06a050 0x7f385c06c510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f387c06cfc0 0x7f387c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f387c14f480 0x7f387c0b36c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 --2- 172.21.15.50:0/2855677992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3864003ef0 0x7f387c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 >> 172.21.15.50:0/2855677992 conn(0x7f387c06fea0 msgr2=0x7f387c1506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 shutdown_connections 2023-10-19T10:37:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.301+0000 7f3880a78640 1 -- 172.21.15.50:0/2855677992 wait complete. 2023-10-19T10:37:08.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:08.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.420+0000 7fca37b18640 1 Processor -- start 2023-10-19T10:37:08.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.421+0000 7fca37b18640 1 -- start start 2023-10-19T10:37:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.421+0000 7fca37b18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b38f0 0x7fca300d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.421+0000 7fca37b18640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fca300b3090 0x7fca30160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.421+0000 7fca37b18640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300af250 con 0x7fca300b3cd0 2023-10-19T10:37:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.421+0000 7fca37b18640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca300b0190 con 0x7fca300b3090 2023-10-19T10:37:08.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.421+0000 7fca37b18640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca300af510 con 0x7fca300b38f0 2023-10-19T10:37:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fca300b3090 0x7fca30160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fca300b3090 0x7fca30160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38510/0 (socket says 172.21.15.50:38510) 2023-10-19T10:37:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 -- 172.21.15.50:0/3297480871 learned_addr learned my addr 172.21.15.50:0/3297480871 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca358b4640 1 --2- 172.21.15.50:0/3297480871 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b38f0 0x7fca300d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 -- 172.21.15.50:0/3297480871 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b38f0 msgr2=0x7fca300d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 --2- 172.21.15.50:0/3297480871 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b38f0 0x7fca300d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 -- 172.21.15.50:0/3297480871 >> v1:172.21.15.183:6789/0 conn(0x7fca300b3cd0 legacy=0x7fca300b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca350b3640 1 -- 172.21.15.50:0/3297480871 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300af7f0 con 0x7fca300b3090 2023-10-19T10:37:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.422+0000 7fca348b2640 1 -- 172.21.15.50:0/3297480871 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958831207 0 0) 0x7fca300af250 con 0x7fca300b3cd0 2023-10-19T10:37:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca350b3640 1 --2- 172.21.15.50:0/3297480871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b3090 0x7fca30160b00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca20009870 tx=0x7fca2002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0f13ab316ee0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca348b2640 1 -- 172.21.15.50:0/3297480871 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20030840 con 0x7fca300b3090 2023-10-19T10:37:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca348b2640 1 -- 172.21.15.50:0/3297480871 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca200309a0 con 0x7fca300b3090 2023-10-19T10:37:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca348b2640 1 -- 172.21.15.50:0/3297480871 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20030d30 con 0x7fca300b3090 2023-10-19T10:37:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca37b18640 1 -- 172.21.15.50:0/3297480871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b3090 msgr2=0x7fca30160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca37b18640 1 --2- 172.21.15.50:0/3297480871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b3090 0x7fca30160b00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca20009870 tx=0x7fca2002b980 comp rx=0 tx=0).stop 2023-10-19T10:37:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca37b18640 1 -- 172.21.15.50:0/3297480871 shutdown_connections 2023-10-19T10:37:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca37b18640 1 --2- 172.21.15.50:0/3297480871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b3090 0x7fca30160b00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca37b18640 1 --2- 172.21.15.50:0/3297480871 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b38f0 0x7fca300d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.423+0000 7fca37b18640 1 -- 172.21.15.50:0/3297480871 >> 172.21.15.50:0/3297480871 conn(0x7fca3009b960 msgr2=0x7fca300d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.424+0000 7fca37b18640 1 -- 172.21.15.50:0/3297480871 shutdown_connections 2023-10-19T10:37:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.424+0000 7fca37b18640 1 -- 172.21.15.50:0/3297480871 wait complete. 2023-10-19T10:37:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 Processor -- start 2023-10-19T10:37:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 -- start start 2023-10-19T10:37:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b3090 0x7fca3015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 0x7fca30159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca300b3cd0 0x7fca30155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300cab20 con 0x7fca300b38f0 2023-10-19T10:37:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300cacb0 con 0x7fca300b3cd0 2023-10-19T10:37:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca37b18640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015a360 con 0x7fca300b3090 2023-10-19T10:37:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca350b3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 0x7fca30159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca350b3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 0x7fca30159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38518/0 (socket says 172.21.15.50:38518) 2023-10-19T10:37:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.425+0000 7fca350b3640 1 -- 172.21.15.50:0/3071506926 learned_addr learned my addr 172.21.15.50:0/3071506926 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca358b4640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b3090 0x7fca3015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca360b5640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca300b3cd0 0x7fca30155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca350b3640 1 -- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b3090 msgr2=0x7fca3015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca350b3640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b3090 0x7fca3015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca350b3640 1 -- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca300b3cd0 msgr2=0x7fca30155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca350b3640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca300b3cd0 0x7fca30155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca350b3640 1 -- 172.21.15.50:0/3071506926 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca200094d0 con 0x7fca300b38f0 2023-10-19T10:37:08.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca350b3640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 0x7fca30159e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca20000c00 tx=0x7fca20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2003e040 con 0x7fca300b38f0 2023-10-19T10:37:08.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20039c90 con 0x7fca300b38f0 2023-10-19T10:37:08.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3015a590 con 0x7fca300b38f0 2023-10-19T10:37:08.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20004030 con 0x7fca300b38f0 2023-10-19T10:37:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.426+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3015ab50 con 0x7fca300b38f0 2023-10-19T10:37:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.427+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca300cae20 con 0x7fca300b38f0 2023-10-19T10:37:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.428+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fca20004190 con 0x7fca300b38f0 2023-10-19T10:37:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.428+0000 7fca267fc640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fca1406a050 0x7fca1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.428+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fca200aaf70 con 0x7fca300b38f0 2023-10-19T10:37:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.428+0000 7fca358b4640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fca1406a050 0x7fca1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.429+0000 7fca358b4640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fca1406a050 0x7fca1406c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca300ce340 tx=0x7fca2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.433+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fca200dd940 con 0x7fca300b38f0 2023-10-19T10:37:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.607+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca300afaa0 con 0x7fca300b38f0 2023-10-19T10:37:08.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.607+0000 7fca267fc640 1 -- 172.21.15.50:0/3071506926 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca200340a0 con 0x7fca300b38f0 2023-10-19T10:37:08.610 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:08.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fca1406a050 msgr2=0x7fca1406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fca1406a050 0x7fca1406c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca300ce340 tx=0x7fca2c008040 comp rx=0 tx=0).stop 2023-10-19T10:37:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 msgr2=0x7fca30159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 0x7fca30159e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca20000c00 tx=0x7fca20004730 comp rx=0 tx=0).stop 2023-10-19T10:37:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 shutdown_connections 2023-10-19T10:37:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fca1406a050 0x7fca1406c510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca300b3cd0 0x7fca30155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca300b38f0 0x7fca30159e20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 --2- 172.21.15.50:0/3071506926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fca300b3090 0x7fca3015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 >> 172.21.15.50:0/3071506926 conn(0x7fca3009b960 msgr2=0x7fca300cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 shutdown_connections 2023-10-19T10:37:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.609+0000 7fca37b18640 1 -- 172.21.15.50:0/3071506926 wait complete. 2023-10-19T10:37:08.622 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.729+0000 7f9281d6e640 1 Processor -- start 2023-10-19T10:37:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.729+0000 7f9281d6e640 1 -- start start 2023-10-19T10:37:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f9281d6e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c159300 0x7f927c071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f9281d6e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f927c1596e0 0x7f927c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f9281d6e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c15a980 con 0x7f927c0cdc00 2023-10-19T10:37:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f9281d6e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c0ce3c0 con 0x7f927c1596e0 2023-10-19T10:37:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f9281d6e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0ce6a0 con 0x7f927c159300 2023-10-19T10:37:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f927affd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f927c1596e0 0x7f927c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f927affd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f927c1596e0 0x7f927c15a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38534/0 (socket says 172.21.15.50:38534) 2023-10-19T10:37:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f927bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f927c0cdc00 0x7f927c070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38362/0 (socket says 172.21.15.50:38362) 2023-10-19T10:37:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.730+0000 7f927affd640 1 -- 172.21.15.50:0/3280705207 learned_addr learned my addr 172.21.15.50:0/3280705207 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927b7fe640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c159300 0x7f927c071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927affd640 1 -- 172.21.15.50:0/3280705207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c159300 msgr2=0x7f927c071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927affd640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c159300 0x7f927c071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927affd640 1 -- 172.21.15.50:0/3280705207 >> v1:172.21.15.50:6789/0 conn(0x7f927c0cdc00 legacy=0x7f927c070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927affd640 1 -- 172.21.15.50:0/3280705207 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c159c40 con 0x7f927c1596e0 2023-10-19T10:37:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927a7fc640 1 -- 172.21.15.50:0/3280705207 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562586899 0 0) 0x7f9274001490 con 0x7f927c0cdc00 2023-10-19T10:37:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927b7fe640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c159300 0x7f927c071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.731+0000 7f927affd640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1596e0 0x7f927c15a440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f927000bb10 tx=0x7f927002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f80f677799e1ab server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f927a7fc640 1 -- 172.21.15.50:0/3280705207 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270036440 con 0x7f927c1596e0 2023-10-19T10:37:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f927a7fc640 1 -- 172.21.15.50:0/3280705207 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92700365a0 con 0x7f927c1596e0 2023-10-19T10:37:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f927a7fc640 1 -- 172.21.15.50:0/3280705207 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270036930 con 0x7f927c1596e0 2023-10-19T10:37:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 -- 172.21.15.50:0/3280705207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1596e0 msgr2=0x7f927c15a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1596e0 0x7f927c15a440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f927000bb10 tx=0x7f927002e150 comp rx=0 tx=0).stop 2023-10-19T10:37:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 -- 172.21.15.50:0/3280705207 shutdown_connections 2023-10-19T10:37:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1596e0 0x7f927c15a440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 --2- 172.21.15.50:0/3280705207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c159300 0x7f927c071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 -- 172.21.15.50:0/3280705207 >> 172.21.15.50:0/3280705207 conn(0x7f927c0cbc00 msgr2=0x7f927c0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 -- 172.21.15.50:0/3280705207 shutdown_connections 2023-10-19T10:37:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.732+0000 7f9281d6e640 1 -- 172.21.15.50:0/3280705207 wait complete. 2023-10-19T10:37:08.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.733+0000 7f9281d6e640 1 Processor -- start 2023-10-19T10:37:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 -- start start 2023-10-19T10:37:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 0x7f927c0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f927c159300 0x7f927c0c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c1596e0 0x7f927c0c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0c87c0 con 0x7f927c0cdc00 2023-10-19T10:37:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0c8200 con 0x7f927c159300 2023-10-19T10:37:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f9281d6e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0c84e0 con 0x7f927c1596e0 2023-10-19T10:37:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 0x7f927c0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 0x7f927c0b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38542/0 (socket says 172.21.15.50:38542) 2023-10-19T10:37:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.734+0000 7f927b7fe640 1 -- 172.21.15.50:0/1396937340 learned_addr learned my addr 172.21.15.50:0/1396937340 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927affd640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f927c159300 0x7f927c0c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927bfff640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c1596e0 0x7f927c0c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927b7fe640 1 -- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c1596e0 msgr2=0x7f927c0c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927b7fe640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c1596e0 0x7f927c0c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927b7fe640 1 -- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f927c159300 msgr2=0x7f927c0c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927b7fe640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f927c159300 0x7f927c0c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927b7fe640 1 -- 172.21.15.50:0/1396937340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927000b770 con 0x7f927c0cdc00 2023-10-19T10:37:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f927b7fe640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 0x7f927c0b4200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c7f0 tx=0x7f926c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c007a30 con 0x7f927c0cdc00 2023-10-19T10:37:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c007b90 con 0x7f927c0cdc00 2023-10-19T10:37:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c0b1e60 con 0x7f927c0cdc00 2023-10-19T10:37:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00ce40 con 0x7f927c0cdc00 2023-10-19T10:37:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.735+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c0b4740 con 0x7f927c0cdc00 2023-10-19T10:37:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.736+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c04e420 con 0x7f927c0cdc00 2023-10-19T10:37:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.737+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f926c0113f0 con 0x7f927c0cdc00 2023-10-19T10:37:08.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.737+0000 7f9263fff640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f926806a050 0x7f926806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:08.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.737+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f926c088a60 con 0x7f927c0cdc00 2023-10-19T10:37:08.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.737+0000 7f927affd640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f926806a050 0x7f926806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:08.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.738+0000 7f927affd640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f926806a050 0x7f926806c510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9270038000 tx=0x7f9270032000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:08.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.742+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f926c053e60 con 0x7f927c0cdc00 2023-10-19T10:37:08.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.919+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f927c0b5100 con 0x7f927c0cdc00 2023-10-19T10:37:08.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.920+0000 7f9263fff640 1 -- 172.21.15.50:0/1396937340 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f926c057480 con 0x7f927c0cdc00 2023-10-19T10:37:08.923 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:08.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f926806a050 msgr2=0x7f926806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f926806a050 0x7f926806c510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9270038000 tx=0x7f9270032000 comp rx=0 tx=0).stop 2023-10-19T10:37:08.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 msgr2=0x7f927c0b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:08.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 0x7f927c0b4200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c7f0 tx=0x7f926c00cbf0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 shutdown_connections 2023-10-19T10:37:08.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f926806a050 0x7f926806c510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f927c1596e0 0x7f927c0c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f927c159300 0x7f927c0c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 --2- 172.21.15.50:0/1396937340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c0cdc00 0x7f927c0b4200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.921+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 >> 172.21.15.50:0/1396937340 conn(0x7f927c0cbc00 msgr2=0x7f927c06eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.922+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 shutdown_connections 2023-10-19T10:37:08.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:08.922+0000 7f9281d6e640 1 -- 172.21.15.50:0/1396937340 wait complete. 2023-10-19T10:37:08.935 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:37:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.039+0000 7f631f89d640 1 Processor -- start 2023-10-19T10:37:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.040+0000 7f631f89d640 1 -- start start 2023-10-19T10:37:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.040+0000 7f631f89d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f631815c570 0x7f63180d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631f89d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f631815c950 0x7f63180cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631f89d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63180cde60 con 0x7f631806b760 2023-10-19T10:37:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631f89d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631806aa30 con 0x7f631815c570 2023-10-19T10:37:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631f89d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631806b2d0 con 0x7f631815c950 2023-10-19T10:37:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f631815c570 0x7f63180d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f631815c570 0x7f63180d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38550/0 (socket says 172.21.15.50:38550) 2023-10-19T10:37:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631de3a640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f631806b760 0x7f63180d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38376/0 (socket says 172.21.15.50:38376) 2023-10-19T10:37:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 -- 172.21.15.50:0/3212696685 learned_addr learned my addr 172.21.15.50:0/3212696685 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631ce38640 1 --2- 172.21.15.50:0/3212696685 >> v2:172.21.15.183:3300/0 conn(0x7f631815c950 0x7f63180cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 -- 172.21.15.50:0/3212696685 >> v2:172.21.15.183:3300/0 conn(0x7f631815c950 msgr2=0x7f63180cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 --2- 172.21.15.50:0/3212696685 >> v2:172.21.15.183:3300/0 conn(0x7f631815c950 0x7f63180cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 -- 172.21.15.50:0/3212696685 >> v1:172.21.15.50:6789/0 conn(0x7f631806b760 legacy=0x7f63180d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f630ffff640 1 -- 172.21.15.50:0/3212696685 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578578068 0 0) 0x7f63180cde60 con 0x7f631806b760 2023-10-19T10:37:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.041+0000 7f631d639640 1 -- 172.21.15.50:0/3212696685 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180ce090 con 0x7f631815c570 2023-10-19T10:37:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631d639640 1 --2- 172.21.15.50:0/3212696685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631815c570 0x7f63180d0e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63180b87c0 tx=0x7f631002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2ec2c3692f069b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f630ffff640 1 -- 172.21.15.50:0/3212696685 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310004230 con 0x7f631815c570 2023-10-19T10:37:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f630ffff640 1 -- 172.21.15.50:0/3212696685 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6310004390 con 0x7f631815c570 2023-10-19T10:37:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f630ffff640 1 -- 172.21.15.50:0/3212696685 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310003810 con 0x7f631815c570 2023-10-19T10:37:09.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631f89d640 1 -- 172.21.15.50:0/3212696685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631815c570 msgr2=0x7f63180d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:09.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631f89d640 1 --2- 172.21.15.50:0/3212696685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631815c570 0x7f63180d0e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63180b87c0 tx=0x7f631002b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631f89d640 1 -- 172.21.15.50:0/3212696685 shutdown_connections 2023-10-19T10:37:09.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631f89d640 1 --2- 172.21.15.50:0/3212696685 >> v2:172.21.15.183:3300/0 conn(0x7f631815c950 0x7f63180cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631f89d640 1 --2- 172.21.15.50:0/3212696685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631815c570 0x7f63180d0e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.042+0000 7f631f89d640 1 -- 172.21.15.50:0/3212696685 >> 172.21.15.50:0/3212696685 conn(0x7f6318089ff0 msgr2=0x7f631809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:09.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.043+0000 7f631f89d640 1 -- 172.21.15.50:0/3212696685 shutdown_connections 2023-10-19T10:37:09.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.043+0000 7f631f89d640 1 -- 172.21.15.50:0/3212696685 wait complete. 2023-10-19T10:37:09.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.044+0000 7f631f89d640 1 Processor -- start 2023-10-19T10:37:09.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.044+0000 7f631f89d640 1 -- start start 2023-10-19T10:37:09.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.044+0000 7f631f89d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 0x7f6318144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:09.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.044+0000 7f631f89d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631815c570 0x7f63180b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:09.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.044+0000 7f631f89d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f631815c950 0x7f63180b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:09.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631f89d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63180c97e0 con 0x7f631806b760 2023-10-19T10:37:09.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631f89d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318156bb0 con 0x7f631815c570 2023-10-19T10:37:09.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631f89d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6318156e90 con 0x7f631815c950 2023-10-19T10:37:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 0x7f6318144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 0x7f6318144fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38552/0 (socket says 172.21.15.50:38552) 2023-10-19T10:37:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 -- 172.21.15.50:0/3210709075 learned_addr learned my addr 172.21.15.50:0/3210709075 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631ce38640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631815c570 0x7f63180b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631de3a640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f631815c950 0x7f63180b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 -- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f631815c950 msgr2=0x7f63180b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f631815c950 0x7f63180b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 -- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631815c570 msgr2=0x7f63180b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631815c570 0x7f63180b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 -- 172.21.15.50:0/3210709075 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310009460 con 0x7f631806b760 2023-10-19T10:37:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631d639640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 0x7f6318144fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6310000c00 tx=0x7f63100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310003c40 con 0x7f631806b760 2023-10-19T10:37:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631003f070 con 0x7f631806b760 2023-10-19T10:37:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310004500 con 0x7f631806b760 2023-10-19T10:37:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.045+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6318157170 con 0x7f631806b760 2023-10-19T10:37:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.046+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6318145520 con 0x7f631806b760 2023-10-19T10:37:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.046+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63180c9950 con 0x7f631806b760 2023-10-19T10:37:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.047+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6310041030 con 0x7f631806b760 2023-10-19T10:37:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.047+0000 7f630dffb640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f62fc06a050 0x7f62fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.047+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63100aaa10 con 0x7f631806b760 2023-10-19T10:37:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.047+0000 7f631ce38640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f62fc06a050 0x7f62fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.048+0000 7f631ce38640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f62fc06a050 0x7f62fc06c510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6318072a30 tx=0x7f6308008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.051+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6310075ea0 con 0x7f631806b760 2023-10-19T10:37:09.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.239+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6318073990 con 0x7f631806b760 2023-10-19T10:37:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.239+0000 7f630dffb640 1 -- 172.21.15.50:0/3210709075 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f63100794c0 con 0x7f631806b760 2023-10-19T10:37:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:37:09.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f62fc06a050 msgr2=0x7f62fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f62fc06a050 0x7f62fc06c510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6318072a30 tx=0x7f6308008040 comp rx=0 tx=0).stop 2023-10-19T10:37:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 msgr2=0x7f6318144fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 0x7f6318144fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6310000c00 tx=0x7f63100046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 shutdown_connections 2023-10-19T10:37:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f62fc06a050 0x7f62fc06c510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f631815c950 0x7f63180b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631815c570 0x7f63180b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 --2- 172.21.15.50:0/3210709075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631806b760 0x7f6318144fe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 >> 172.21.15.50:0/3210709075 conn(0x7f6318089ff0 msgr2=0x7f631815aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 shutdown_connections 2023-10-19T10:37:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:09.241+0000 7f631f89d640 1 -- 172.21.15.50:0/3210709075 wait complete. 2023-10-19T10:37:09.254 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-19T10:37:09.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:09.255 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-19T10:37:09.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[94800]: fuse finished with error 0 and tester_r 0 2023-10-19T10:37:09.286 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:37:15.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:37:15.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:15.389 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-19T10:37:15.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:15.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 Processor -- start 2023-10-19T10:37:15.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 -- start start 2023-10-19T10:37:15.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1c200d0e30 0x7f1c2015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 0x7f1c200d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2006aa30 con 0x7f1c2015c570 2023-10-19T10:37:15.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2015ce90 con 0x7f1c200d0e30 2023-10-19T10:37:15.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.572+0000 7f1c26df2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2006b2d0 con 0x7f1c200cd920 2023-10-19T10:37:15.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c24b8e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1c2015c570 0x7f1c2006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38382/0 (socket says 172.21.15.50:38382) 2023-10-19T10:37:15.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c24b8e640 1 -- 172.21.15.50:0/2619075853 learned_addr learned my addr 172.21.15.50:0/2619075853 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:15.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1ffff640 1 --2- 172.21.15.50:0/2619075853 >> v2:172.21.15.183:3300/0 conn(0x7f1c200d0e30 0x7f1c2015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c2538f640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 0x7f1c200d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779715249 0 0) 0x7f1c2006aa30 con 0x7f1c2015c570 2023-10-19T10:37:15.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c0c003520 con 0x7f1c2015c570 2023-10-19T10:37:15.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91702738 0 0) 0x7f1c0c003520 con 0x7f1c2015c570 2023-10-19T10:37:15.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 msgr2=0x7f1c200d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:15.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 0x7f1c200d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 >> v2:172.21.15.183:3300/0 conn(0x7f1c200d0e30 msgr2=0x7f1c2015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:15.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 --2- 172.21.15.50:0/2619075853 >> v2:172.21.15.183:3300/0 conn(0x7f1c200d0e30 0x7f1c2015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200d1270 con 0x7f1c2015c570 2023-10-19T10:37:15.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f1c18001b70 con 0x7f1c2015c570 2023-10-19T10:37:15.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 >> v1:172.21.15.50:6789/0 conn(0x7f1c2015c570 legacy=0x7f1c2006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c0c003970 0x7f1c0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 0x7f1c0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c0c004d00 0x7f1c0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2015ce90 con 0x7f1c0c004d00 2023-10-19T10:37:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2006b2d0 con 0x7f1c0c003970 2023-10-19T10:37:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c00b980 con 0x7f1c200cd920 2023-10-19T10:37:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c2538f640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c0c004d00 0x7f1c0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c1ffff640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 0x7f1c0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.573+0000 7f1c24b8e640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c0c003970 0x7f1c0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.574+0000 7f1c2538f640 1 -- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 msgr2=0x7f1c0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.574+0000 7f1c2538f640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c200cd920 0x7f1c0c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.574+0000 7f1c2538f640 1 -- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c0c003970 msgr2=0x7f1c0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:15.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.574+0000 7f1c2538f640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c0c003970 0x7f1c0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.574+0000 7f1c2538f640 1 -- 172.21.15.50:0/2619075853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c003520 con 0x7f1c0c004d00 2023-10-19T10:37:15.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.574+0000 7f1c2538f640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c0c004d00 0x7f1c0c005180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ac0 tx=0x7f1c0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961fd6492fddf348 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:15.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1002c6b0 con 0x7f1c0c004d00 2023-10-19T10:37:15.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1002c810 con 0x7f1c0c004d00 2023-10-19T10:37:15.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c1f7fe640 1 -- 172.21.15.50:0/2619075853 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1002cba0 con 0x7f1c0c004d00 2023-10-19T10:37:15.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c26df2640 1 -- 172.21.15.50:0/2619075853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c0c004d00 msgr2=0x7f1c0c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c26df2640 1 --2- 172.21.15.50:0/2619075853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c0c004d00 0x7f1c0c005180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ac0 tx=0x7f1c0c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c2538f640 1 -- 172.21.15.50:0/2619075853 reap_dead start 2023-10-19T10:37:15.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c26df2640 1 -- 172.21.15.50:0/2619075853 shutdown_connections 2023-10-19T10:37:15.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c26df2640 1 -- 172.21.15.50:0/2619075853 >> 172.21.15.50:0/2619075853 conn(0x7f1c20089ff0 msgr2=0x7f1c2009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:15.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c26df2640 1 -- 172.21.15.50:0/2619075853 shutdown_connections 2023-10-19T10:37:15.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.575+0000 7f1c26df2640 1 -- 172.21.15.50:0/2619075853 wait complete. 2023-10-19T10:37:15.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.576+0000 7f1c26df2640 1 Processor -- start 2023-10-19T10:37:15.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.576+0000 7f1c26df2640 1 -- start start 2023-10-19T10:37:15.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.577+0000 7f1c26df2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 0x7f1c200cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.577+0000 7f1c26df2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c2015a080 0x7f1c200b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.577+0000 7f1c26df2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c200b12f0 0x7f1c201e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.577+0000 7f1c26df2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200b1770 con 0x7f1c2014d920 2023-10-19T10:37:15.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.577+0000 7f1c26df2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200cc560 con 0x7f1c200b12f0 2023-10-19T10:37:15.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.577+0000 7f1c26df2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c6750 con 0x7f1c2015a080 2023-10-19T10:37:15.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 0x7f1c200cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 0x7f1c200cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38582/0 (socket says 172.21.15.50:38582) 2023-10-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 -- 172.21.15.50:0/1009444508 learned_addr learned my addr 172.21.15.50:0/1009444508 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c2538f640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c200b12f0 0x7f1c201e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c1ffff640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c2015a080 0x7f1c200b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 -- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c2015a080 msgr2=0x7f1c200b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c2015a080 0x7f1c200b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 -- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c200b12f0 msgr2=0x7f1c201e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c200b12f0 0x7f1c201e4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 -- 172.21.15.50:0/1009444508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200cbfa0 con 0x7f1c2014d920 2023-10-19T10:37:15.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c24b8e640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 0x7f1c200cb750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c18001ac0 tx=0x7f1c1800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:15.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800de40 con 0x7f1c2014d920 2023-10-19T10:37:15.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18005b20 con 0x7f1c2014d920 2023-10-19T10:37:15.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c200cc280 con 0x7f1c2014d920 2023-10-19T10:37:15.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180070d0 con 0x7f1c2014d920 2023-10-19T10:37:15.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.578+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c200cbc90 con 0x7f1c2014d920 2023-10-19T10:37:15.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.579+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c200c6240 con 0x7f1c2014d920 2023-10-19T10:37:15.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.580+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c18007230 con 0x7f1c2014d920 2023-10-19T10:37:15.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.580+0000 7f1c1d7fa640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0006a050 0x7f1c0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.580+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1c18088e40 con 0x7f1c2014d920 2023-10-19T10:37:15.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.581+0000 7f1c1ffff640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0006a050 0x7f1c0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.581+0000 7f1c1ffff640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0006a050 0x7f1c0006c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c2014e610 tx=0x7f1c08009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:15.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.585+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1c200c6240 con 0x7f1c2014d920 2023-10-19T10:37:15.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.762+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c200c6240 con 0x7f1c2014d920 2023-10-19T10:37:15.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.764+0000 7f1c1d7fa640 1 -- 172.21.15.50:0/1009444508 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f1c200c6240 con 0x7f1c2014d920 2023-10-19T10:37:15.766 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:15.767 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:15.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.765+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0006a050 msgr2=0x7f1c0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.765+0000 7f1c26df2640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0006a050 0x7f1c0006c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c2014e610 tx=0x7f1c08009190 comp rx=0 tx=0).stop 2023-10-19T10:37:15.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.765+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 msgr2=0x7f1c200cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.765+0000 7f1c26df2640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 0x7f1c200cb750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c18001ac0 tx=0x7f1c1800d520 comp rx=0 tx=0).stop 2023-10-19T10:37:15.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 shutdown_connections 2023-10-19T10:37:15.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0006a050 0x7f1c0006c510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c200b12f0 0x7f1c201e4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c2015a080 0x7f1c200b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 --2- 172.21.15.50:0/1009444508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c2014d920 0x7f1c200cb750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 >> 172.21.15.50:0/1009444508 conn(0x7f1c20089ff0 msgr2=0x7f1c200aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:15.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 shutdown_connections 2023-10-19T10:37:15.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.766+0000 7f1c26df2640 1 -- 172.21.15.50:0/1009444508 wait complete. 2023-10-19T10:37:15.779 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:15.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.883+0000 7f2e8ac34640 1 Processor -- start 2023-10-19T10:37:15.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.883+0000 7f2e8ac34640 1 -- start start 2023-10-19T10:37:15.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.883+0000 7f2e8ac34640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2e8415c570 0x7f2e840d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.883+0000 7f2e8ac34640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2e840cd920 0x7f2e840d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e8ac34640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8406aa30 con 0x7f2e8415c950 2023-10-19T10:37:15.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e8ac34640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8406b2d0 con 0x7f2e840cd920 2023-10-19T10:37:15.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e8ac34640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e840ce090 con 0x7f2e8415c570 2023-10-19T10:37:15.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2e840cd920 0x7f2e840d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2e840cd920 0x7f2e840d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38598/0 (socket says 172.21.15.50:38598) 2023-10-19T10:37:15.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 -- 172.21.15.50:0/840194149 learned_addr learned my addr 172.21.15.50:0/840194149 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:15.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e889d0640 1 --2- 172.21.15.50:0/840194149 >> v2:172.21.15.183:3300/0 conn(0x7f2e8415c570 0x7f2e840d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 -- 172.21.15.50:0/840194149 >> v2:172.21.15.183:3300/0 conn(0x7f2e8415c570 msgr2=0x7f2e840d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:15.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 --2- 172.21.15.50:0/840194149 >> v2:172.21.15.183:3300/0 conn(0x7f2e8415c570 0x7f2e840d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 -- 172.21.15.50:0/840194149 >> v1:172.21.15.183:6789/0 conn(0x7f2e8415c950 legacy=0x7f2e8406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.884+0000 7f2e891d1640 1 -- 172.21.15.50:0/840194149 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8406ac80 con 0x7f2e840cd920 2023-10-19T10:37:15.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e891d1640 1 --2- 172.21.15.50:0/840194149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840cd920 0x7f2e840d26a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e840d2670 tx=0x7f2e7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50273bd73ef7a9f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:15.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e837fe640 1 -- 172.21.15.50:0/840194149 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c02ca80 con 0x7f2e840cd920 2023-10-19T10:37:15.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e837fe640 1 -- 172.21.15.50:0/840194149 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c02cbe0 con 0x7f2e840cd920 2023-10-19T10:37:15.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e837fe640 1 -- 172.21.15.50:0/840194149 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0343f0 con 0x7f2e840cd920 2023-10-19T10:37:15.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e8ac34640 1 -- 172.21.15.50:0/840194149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840cd920 msgr2=0x7f2e840d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/840194149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840cd920 0x7f2e840d26a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e840d2670 tx=0x7f2e7c02bee0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e8ac34640 1 -- 172.21.15.50:0/840194149 shutdown_connections 2023-10-19T10:37:15.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/840194149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e840cd920 0x7f2e840d26a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/840194149 >> v2:172.21.15.183:3300/0 conn(0x7f2e8415c570 0x7f2e840d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.885+0000 7f2e8ac34640 1 -- 172.21.15.50:0/840194149 >> 172.21.15.50:0/840194149 conn(0x7f2e84089ff0 msgr2=0x7f2e8409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:15.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.886+0000 7f2e8ac34640 1 -- 172.21.15.50:0/840194149 shutdown_connections 2023-10-19T10:37:15.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.886+0000 7f2e8ac34640 1 -- 172.21.15.50:0/840194149 wait complete. 2023-10-19T10:37:15.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.887+0000 7f2e8ac34640 1 Processor -- start 2023-10-19T10:37:15.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.887+0000 7f2e8ac34640 1 -- start start 2023-10-19T10:37:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e840cd920 0x7f2e8409c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 0x7f2e8415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e8415c950 0x7f2e8415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8409c970 con 0x7f2e8415c570 2023-10-19T10:37:15.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840ca6d0 con 0x7f2e8415c950 2023-10-19T10:37:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8415b710 con 0x7f2e840cd920 2023-10-19T10:37:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 0x7f2e8415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 0x7f2e8415b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38602/0 (socket says 172.21.15.50:38602) 2023-10-19T10:37:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 -- 172.21.15.50:0/847634965 learned_addr learned my addr 172.21.15.50:0/847634965 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:15.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e891d1640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e8415c950 0x7f2e8415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e889d0640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e840cd920 0x7f2e8409c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 -- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e840cd920 msgr2=0x7f2e8409c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e840cd920 0x7f2e8409c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 -- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e8415c950 msgr2=0x7f2e8415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:15.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e8415c950 0x7f2e8415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:15.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 -- 172.21.15.50:0/847634965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009b30 con 0x7f2e8415c570 2023-10-19T10:37:15.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e889d0640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e840cd920 0x7f2e8409c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:15.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e83fff640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 0x7f2e8415b1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c450 tx=0x7f2e6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:15.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c0052d0 con 0x7f2e8415c570 2023-10-19T10:37:15.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c005430 con 0x7f2e8415c570 2023-10-19T10:37:15.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e840caa70 con 0x7f2e8415c570 2023-10-19T10:37:15.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c012870 con 0x7f2e8415c570 2023-10-19T10:37:15.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.888+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e840cb030 con 0x7f2e8415c570 2023-10-19T10:37:15.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.889+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e84073990 con 0x7f2e8415c570 2023-10-19T10:37:15.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.890+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e6c0129d0 con 0x7f2e8415c570 2023-10-19T10:37:15.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.890+0000 7f2e817fa640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e6406a050 0x7f2e6406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:15.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.890+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2e6c0884c0 con 0x7f2e8415c570 2023-10-19T10:37:15.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.890+0000 7f2e889d0640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e6406a050 0x7f2e6406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:15.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.891+0000 7f2e889d0640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e6406a050 0x7f2e6406c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009ab0 tx=0x7f2e74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:15.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:15.895+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2e6c053820 con 0x7f2e8415c570 2023-10-19T10:37:16.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.069+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e840d3480 con 0x7f2e8415c570 2023-10-19T10:37:16.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.070+0000 7f2e817fa640 1 -- 172.21.15.50:0/847634965 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f2e6c056e40 con 0x7f2e8415c570 2023-10-19T10:37:16.073 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:16.073 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:16.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e6406a050 msgr2=0x7f2e6406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:16.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e6406a050 0x7f2e6406c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009ab0 tx=0x7f2e74008040 comp rx=0 tx=0).stop 2023-10-19T10:37:16.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 msgr2=0x7f2e8415b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:16.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 0x7f2e8415b1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c450 tx=0x7f2e6c00c910 comp rx=0 tx=0).stop 2023-10-19T10:37:16.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 shutdown_connections 2023-10-19T10:37:16.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e6406a050 0x7f2e6406c510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e8415c950 0x7f2e8415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e8415c570 0x7f2e8415b1d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 --2- 172.21.15.50:0/847634965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e840cd920 0x7f2e8409c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 >> 172.21.15.50:0/847634965 conn(0x7f2e84089ff0 msgr2=0x7f2e840b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:16.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 shutdown_connections 2023-10-19T10:37:16.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.072+0000 7f2e8ac34640 1 -- 172.21.15.50:0/847634965 wait complete. 2023-10-19T10:37:16.085 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-19T10:37:16.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.192+0000 7ff8db7c7640 1 Processor -- start 2023-10-19T10:37:16.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.192+0000 7ff8db7c7640 1 -- start start 2023-10-19T10:37:16.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8db7c7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff8d40b2420 0x7ff8d415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:16.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8db7c7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d4160d70 0x7ff8d415d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:16.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8db7c7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d415d8e0 con 0x7ff8d4145640 2023-10-19T10:37:16.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8db7c7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d4160290 con 0x7ff8d40b2420 2023-10-19T10:37:16.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8db7c7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4160570 con 0x7ff8d4160d70 2023-10-19T10:37:16.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8d8d62640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff8d40b2420 0x7ff8d415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:16.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8d8d62640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff8d40b2420 0x7ff8d415cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38604/0 (socket says 172.21.15.50:38604) 2023-10-19T10:37:16.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8d9563640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff8d4145640 0x7ff8d40b1d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:38398/0 (socket says 172.21.15.50:38398) 2023-10-19T10:37:16.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.193+0000 7ff8d8d62640 1 -- 172.21.15.50:0/1813759781 learned_addr learned my addr 172.21.15.50:0/1813759781 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:16.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d9d64640 1 --2- 172.21.15.50:0/1813759781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d4160d70 0x7ff8d415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:16.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d8d62640 1 -- 172.21.15.50:0/1813759781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d4160d70 msgr2=0x7ff8d415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:16.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d8d62640 1 --2- 172.21.15.50:0/1813759781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d4160d70 0x7ff8d415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d8d62640 1 -- 172.21.15.50:0/1813759781 >> v1:172.21.15.50:6789/0 conn(0x7ff8d4145640 legacy=0x7ff8d40b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:16.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d8d62640 1 -- 172.21.15.50:0/1813759781 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4160850 con 0x7ff8d40b2420 2023-10-19T10:37:16.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d3fff640 1 -- 172.21.15.50:0/1813759781 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747021146 0 0) 0x7ff8d415d8e0 con 0x7ff8d4145640 2023-10-19T10:37:16.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.194+0000 7ff8d8d62640 1 --2- 172.21.15.50:0/1813759781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d40b2420 0x7ff8d415cb40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009870 tx=0x7ff8c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a7af897b4fd8fa server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:16.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8d3fff640 1 -- 172.21.15.50:0/1813759781 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8030840 con 0x7ff8d40b2420 2023-10-19T10:37:16.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8d3fff640 1 -- 172.21.15.50:0/1813759781 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80309a0 con 0x7ff8d40b2420 2023-10-19T10:37:16.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8d3fff640 1 -- 172.21.15.50:0/1813759781 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8030d30 con 0x7ff8d40b2420 2023-10-19T10:37:16.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 -- 172.21.15.50:0/1813759781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d40b2420 msgr2=0x7ff8d415cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:16.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/1813759781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d40b2420 0x7ff8d415cb40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009870 tx=0x7ff8c802b980 comp rx=0 tx=0).stop 2023-10-19T10:37:16.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 -- 172.21.15.50:0/1813759781 shutdown_connections 2023-10-19T10:37:16.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/1813759781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d4160d70 0x7ff8d415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/1813759781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d40b2420 0x7ff8d415cb40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 -- 172.21.15.50:0/1813759781 >> 172.21.15.50:0/1813759781 conn(0x7ff8d40ccd10 msgr2=0x7ff8d40cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:16.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 -- 172.21.15.50:0/1813759781 shutdown_connections 2023-10-19T10:37:16.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.195+0000 7ff8db7c7640 1 -- 172.21.15.50:0/1813759781 wait complete. 2023-10-19T10:37:16.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.196+0000 7ff8db7c7640 1 Processor -- start 2023-10-19T10:37:16.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.196+0000 7ff8db7c7640 1 -- start start 2023-10-19T10:37:16.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8db7c7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d40b2420 0x7ff8d40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:16.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8db7c7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 0x7ff8d40c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:16.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8db7c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8d4160d70 0x7ff8d415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:16.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8db7c7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d40c4c00 con 0x7ff8d4145640 2023-10-19T10:37:16.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8db7c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d40b3580 con 0x7ff8d4160d70 2023-10-19T10:37:16.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8db7c7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d40c98c0 con 0x7ff8d40b2420 2023-10-19T10:37:16.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8d8d62640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 0x7ff8d40c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:16.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8d8d62640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 0x7ff8d40c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38608/0 (socket says 172.21.15.50:38608) 2023-10-19T10:37:16.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8d8d62640 1 -- 172.21.15.50:0/3264492347 learned_addr learned my addr 172.21.15.50:0/3264492347 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:16.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.197+0000 7ff8d9d64640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8d4160d70 0x7ff8d415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:16.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d9563640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d40b2420 0x7ff8d40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:16.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d8d62640 1 -- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d40b2420 msgr2=0x7ff8d40c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:16.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d8d62640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d40b2420 0x7ff8d40c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d8d62640 1 -- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8d4160d70 msgr2=0x7ff8d415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:16.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d8d62640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8d4160d70 0x7ff8d415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:16.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d8d62640 1 -- 172.21.15.50:0/3264492347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80094d0 con 0x7ff8d4145640 2023-10-19T10:37:16.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d8d62640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 0x7ff8d40c9050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8000c00 tx=0x7ff8c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:16.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c803e070 con 0x7ff8d4145640 2023-10-19T10:37:16.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c8004030 con 0x7ff8d4145640 2023-10-19T10:37:16.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d40b2f30 con 0x7ff8d4145640 2023-10-19T10:37:16.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80043c0 con 0x7ff8d4145640 2023-10-19T10:37:16.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.198+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d40c9ba0 con 0x7ff8d4145640 2023-10-19T10:37:16.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.199+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d404e420 con 0x7ff8d4145640 2023-10-19T10:37:16.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.200+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff8c8042070 con 0x7ff8d4145640 2023-10-19T10:37:16.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.200+0000 7ff8d1ffb640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff8c006a050 0x7ff8c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:16.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.200+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff8c80aa620 con 0x7ff8d4145640 2023-10-19T10:37:16.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.200+0000 7ff8d9563640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff8c006a050 0x7ff8c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:16.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.201+0000 7ff8d9563640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff8c006a050 0x7ff8c006c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc000f60 tx=0x7ff8cc007b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:16.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.204+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff8c8075bd0 con 0x7ff8d4145640 2023-10-19T10:37:16.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:16.418+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff8d406e560 con 0x7ff8d4145640 2023-10-19T10:37:17.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.122+0000 7ff8d1ffb640 1 -- 172.21.15.50:0/3264492347 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff8c80791f0 con 0x7ff8d4145640 2023-10-19T10:37:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff8c006a050 msgr2=0x7ff8c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff8c006a050 0x7ff8c006c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc000f60 tx=0x7ff8cc007b90 comp rx=0 tx=0).stop 2023-10-19T10:37:17.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 msgr2=0x7ff8d40c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 0x7ff8d40c9050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8000c00 tx=0x7ff8c80046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 shutdown_connections 2023-10-19T10:37:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff8c006a050 0x7ff8c006c510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8d4160d70 0x7ff8d415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4145640 0x7ff8d40c9050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 --2- 172.21.15.50:0/3264492347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8d40b2420 0x7ff8d40c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 >> 172.21.15.50:0/3264492347 conn(0x7ff8d40ccd10 msgr2=0x7ff8d40b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 shutdown_connections 2023-10-19T10:37:17.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.125+0000 7ff8db7c7640 1 -- 172.21.15.50:0/3264492347 wait complete. 2023-10-19T10:37:17.141 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-19T10:37:17.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-19T10:37:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.254+0000 7f2729d85640 1 Processor -- start 2023-10-19T10:37:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.254+0000 7f2729d85640 1 -- start start 2023-10-19T10:37:17.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2729d85640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f27240b4070 0x7f27240b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:17.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2729d85640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f272415c960 0x7f27240d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2729d85640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27240b4990 con 0x7f272415cdf0 2023-10-19T10:37:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2729d85640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272415d280 con 0x7f27240b4070 2023-10-19T10:37:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2729d85640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272415d560 con 0x7f272415c960 2023-10-19T10:37:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f27240b4070 0x7f27240b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f27240b4070 0x7f27240b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38744/0 (socket says 172.21.15.50:38744) 2023-10-19T10:37:17.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 -- 172.21.15.50:0/3452629701 learned_addr learned my addr 172.21.15.50:0/3452629701 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:17.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2723fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f272415cdf0 0x7f2724160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33482/0 (socket says 172.21.15.50:33482) 2023-10-19T10:37:17.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f2722ffd640 1 --2- 172.21.15.50:0/3452629701 >> v2:172.21.15.183:3300/0 conn(0x7f272415c960 0x7f27240d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:17.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 -- 172.21.15.50:0/3452629701 >> v2:172.21.15.183:3300/0 conn(0x7f272415c960 msgr2=0x7f27240d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:17.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 --2- 172.21.15.50:0/3452629701 >> v2:172.21.15.183:3300/0 conn(0x7f272415c960 0x7f27240d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 -- 172.21.15.50:0/3452629701 >> v1:172.21.15.50:6789/0 conn(0x7f272415cdf0 legacy=0x7f2724160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:17.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27227fc640 1 -- 172.21.15.50:0/3452629701 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391023793 0 0) 0x7f27240b4990 con 0x7f272415cdf0 2023-10-19T10:37:17.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.255+0000 7f27237fe640 1 -- 172.21.15.50:0/3452629701 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272415d840 con 0x7f27240b4070 2023-10-19T10:37:17.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.256+0000 7f27237fe640 1 --2- 172.21.15.50:0/3452629701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240b4450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27240ccb40 tx=0x7f271402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214653109ad96f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.256+0000 7f27227fc640 1 -- 172.21.15.50:0/3452629701 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714004230 con 0x7f27240b4070 2023-10-19T10:37:17.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.256+0000 7f27227fc640 1 -- 172.21.15.50:0/3452629701 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714004390 con 0x7f27240b4070 2023-10-19T10:37:17.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.256+0000 7f27227fc640 1 -- 172.21.15.50:0/3452629701 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714030840 con 0x7f27240b4070 2023-10-19T10:37:17.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 -- 172.21.15.50:0/3452629701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 msgr2=0x7f27240b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:17.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 --2- 172.21.15.50:0/3452629701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240b4450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27240ccb40 tx=0x7f271402b980 comp rx=0 tx=0).stop 2023-10-19T10:37:17.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 -- 172.21.15.50:0/3452629701 shutdown_connections 2023-10-19T10:37:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 --2- 172.21.15.50:0/3452629701 >> v2:172.21.15.183:3300/0 conn(0x7f272415c960 0x7f27240d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 --2- 172.21.15.50:0/3452629701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240b4450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 -- 172.21.15.50:0/3452629701 >> 172.21.15.50:0/3452629701 conn(0x7f272409b970 msgr2=0x7f27240d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 -- 172.21.15.50:0/3452629701 shutdown_connections 2023-10-19T10:37:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.257+0000 7f2729d85640 1 -- 172.21.15.50:0/3452629701 wait complete. 2023-10-19T10:37:17.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.258+0000 7f2729d85640 1 Processor -- start 2023-10-19T10:37:17.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.258+0000 7f2729d85640 1 -- start start 2023-10-19T10:37:17.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2729d85640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:17.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2729d85640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f272415c960 0x7f27241f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:17.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2729d85640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f272415aea0 0x7f27241598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:17.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2729d85640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272415b320 con 0x7f27240b4070 2023-10-19T10:37:17.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2729d85640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272415b950 con 0x7f272415c960 2023-10-19T10:37:17.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2729d85640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27241f5630 con 0x7f272415aea0 2023-10-19T10:37:17.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:17.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38758/0 (socket says 172.21.15.50:38758) 2023-10-19T10:37:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 -- 172.21.15.50:0/237542410 learned_addr learned my addr 172.21.15.50:0/237542410 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2722ffd640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f272415c960 0x7f27241f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2723fff640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f272415aea0 0x7f27241598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 -- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f272415aea0 msgr2=0x7f27241598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f272415aea0 0x7f27241598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 -- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f272415c960 msgr2=0x7f27241f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f272415c960 0x7f27241f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f27237fe640 1 -- 172.21.15.50:0/237542410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714009460 con 0x7f27240b4070 2023-10-19T10:37:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.259+0000 7f2723fff640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f272415aea0 0x7f27241598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.260+0000 7f27237fe640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240c81f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2714009940 tx=0x7f27140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.260+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271403e070 con 0x7f27240b4070 2023-10-19T10:37:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.260+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714004500 con 0x7f27240b4070 2023-10-19T10:37:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.260+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27241f2560 con 0x7f27240b4070 2023-10-19T10:37:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.260+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714030e20 con 0x7f27240b4070 2023-10-19T10:37:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.260+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27241f2b20 con 0x7f27240b4070 2023-10-19T10:37:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.261+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272404e420 con 0x7f27240b4070 2023-10-19T10:37:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.261+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2714003610 con 0x7f27240b4070 2023-10-19T10:37:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.261+0000 7f270bfff640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f271006a050 0x7f271006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.262+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f27140aaa50 con 0x7f27240b4070 2023-10-19T10:37:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.262+0000 7f2722ffd640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f271006a050 0x7f271006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.262+0000 7f2722ffd640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f271006a050 0x7f271006c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f272415faa0 tx=0x7f2718008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.266+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2714075e50 con 0x7f27240b4070 2023-10-19T10:37:17.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:17.486+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f272408c720 con 0x7f27240b4070 2023-10-19T10:37:18.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.124+0000 7f270bfff640 1 -- 172.21.15.50:0/237542410 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2714079470 con 0x7f27240b4070 2023-10-19T10:37:18.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.127+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f271006a050 msgr2=0x7f271006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.127+0000 7f2729d85640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f271006a050 0x7f271006c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f272415faa0 tx=0x7f2718008040 comp rx=0 tx=0).stop 2023-10-19T10:37:18.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.127+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 msgr2=0x7f27240c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.127+0000 7f2729d85640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240c81f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2714009940 tx=0x7f27140046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 shutdown_connections 2023-10-19T10:37:18.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f271006a050 0x7f271006c510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f272415aea0 0x7f27241598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f272415c960 0x7f27241f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 --2- 172.21.15.50:0/237542410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27240b4070 0x7f27240c81f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 >> 172.21.15.50:0/237542410 conn(0x7f272409b970 msgr2=0x7f2724160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:18.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 shutdown_connections 2023-10-19T10:37:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.128+0000 7f2729d85640 1 -- 172.21.15.50:0/237542410 wait complete. 2023-10-19T10:37:18.143 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-19T10:37:18.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:37:18.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.254+0000 7f438d7d0640 1 Processor -- start 2023-10-19T10:37:18.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.254+0000 7f438d7d0640 1 -- start start 2023-10-19T10:37:18.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f438d7d0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f43880d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f438d7d0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f43880cd920 0x7f43880d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f438d7d0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438806aa30 con 0x7f438815c950 2023-10-19T10:37:18.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f438d7d0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438806b2d0 con 0x7f43880cd920 2023-10-19T10:37:18.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f438d7d0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880ce090 con 0x7f438815c570 2023-10-19T10:37:18.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f43877fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f43880cd920 0x7f43880d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f437ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f438815c950 0x7f438806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33494/0 (socket says 172.21.15.50:33494) 2023-10-19T10:37:18.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f437ffff640 1 -- 172.21.15.50:0/2565928631 learned_addr learned my addr 172.21.15.50:0/2565928631 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:18.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f4386ffd640 1 --2- 172.21.15.50:0/2565928631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f43880d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f43877fe640 1 -- 172.21.15.50:0/2565928631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 msgr2=0x7f43880d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:18.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f43877fe640 1 --2- 172.21.15.50:0/2565928631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f43880d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f43877fe640 1 -- 172.21.15.50:0/2565928631 >> v1:172.21.15.50:6789/0 conn(0x7f438815c950 legacy=0x7f438806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.255+0000 7f43877fe640 1 -- 172.21.15.50:0/2565928631 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438806ac80 con 0x7f43880cd920 2023-10-19T10:37:18.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.256+0000 7f43877fe640 1 --2- 172.21.15.50:0/2565928631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cd920 0x7f43880d26a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4380009d90 tx=0x7f438002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ff8119c8ef16cc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.256+0000 7f43867fc640 1 -- 172.21.15.50:0/2565928631 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438002cbc0 con 0x7f43880cd920 2023-10-19T10:37:18.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.256+0000 7f43867fc640 1 -- 172.21.15.50:0/2565928631 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438002cd20 con 0x7f43880cd920 2023-10-19T10:37:18.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.256+0000 7f43867fc640 1 -- 172.21.15.50:0/2565928631 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380034660 con 0x7f43880cd920 2023-10-19T10:37:18.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 -- 172.21.15.50:0/2565928631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cd920 msgr2=0x7f43880d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 --2- 172.21.15.50:0/2565928631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cd920 0x7f43880d26a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4380009d90 tx=0x7f438002be10 comp rx=0 tx=0).stop 2023-10-19T10:37:18.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 -- 172.21.15.50:0/2565928631 shutdown_connections 2023-10-19T10:37:18.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 --2- 172.21.15.50:0/2565928631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43880cd920 0x7f43880d26a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 --2- 172.21.15.50:0/2565928631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f43880d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 -- 172.21.15.50:0/2565928631 >> 172.21.15.50:0/2565928631 conn(0x7f4388089ff0 msgr2=0x7f438809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:18.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 -- 172.21.15.50:0/2565928631 shutdown_connections 2023-10-19T10:37:18.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.257+0000 7f438d7d0640 1 -- 172.21.15.50:0/2565928631 wait complete. 2023-10-19T10:37:18.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.258+0000 7f438d7d0640 1 Processor -- start 2023-10-19T10:37:18.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.258+0000 7f438d7d0640 1 -- start start 2023-10-19T10:37:18.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f438d7d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43880cd920 0x7f43880b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f438d7d0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f438815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f438d7d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 0x7f438815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f438d7d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880af2b0 con 0x7f438815c950 2023-10-19T10:37:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f438d7d0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438815fa80 con 0x7f43880cd920 2023-10-19T10:37:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f438d7d0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880af520 con 0x7f438815c570 2023-10-19T10:37:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 0x7f438815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 0x7f438815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38772/0 (socket says 172.21.15.50:38772) 2023-10-19T10:37:18.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 -- 172.21.15.50:0/148444550 learned_addr learned my addr 172.21.15.50:0/148444550 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:18.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f4386ffd640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43880cd920 0x7f43880b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f437ffff640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f438815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 -- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 msgr2=0x7f438815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f438815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 -- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43880cd920 msgr2=0x7f43880b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43880cd920 0x7f43880b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 -- 172.21.15.50:0/148444550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4380009a70 con 0x7f438815c950 2023-10-19T10:37:18.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.259+0000 7f43877fe640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 0x7f438815d4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4380009d90 tx=0x7f4380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.260+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438003e610 con 0x7f438815c950 2023-10-19T10:37:18.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.260+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438002ce90 con 0x7f438815c950 2023-10-19T10:37:18.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.260+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43880b6120 con 0x7f438815c950 2023-10-19T10:37:18.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.260+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438003e9f0 con 0x7f438815c950 2023-10-19T10:37:18.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.260+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438815f4c0 con 0x7f438815c950 2023-10-19T10:37:18.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.261+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4388073990 con 0x7f438815c950 2023-10-19T10:37:18.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.261+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f438003b050 con 0x7f438815c950 2023-10-19T10:37:18.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.261+0000 7f437f7fe640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f436806a080 0x7f436806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.262+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f43800aac60 con 0x7f438815c950 2023-10-19T10:37:18.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.262+0000 7f4386ffd640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f436806a080 0x7f436806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.262+0000 7f4386ffd640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f436806a080 0x7f436806c540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43880d2fa0 tx=0x7f4378008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.266+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4380076210 con 0x7f438815c950 2023-10-19T10:37:18.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.482+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43880af660 con 0x7f438815c950 2023-10-19T10:37:18.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.482+0000 7f437f7fe640 1 -- 172.21.15.50:0/148444550 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7f4380039020 con 0x7f438815c950 2023-10-19T10:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:37:18.486 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:37:18.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f436806a080 msgr2=0x7f436806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f436806a080 0x7f436806c540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43880d2fa0 tx=0x7f4378008040 comp rx=0 tx=0).stop 2023-10-19T10:37:18.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 msgr2=0x7f438815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 0x7f438815d4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4380009d90 tx=0x7f4380004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:18.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 shutdown_connections 2023-10-19T10:37:18.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f436806a080 0x7f436806c540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f438815c950 0x7f438815d4b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f438815c570 0x7f438815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 --2- 172.21.15.50:0/148444550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43880cd920 0x7f43880b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 >> 172.21.15.50:0/148444550 conn(0x7f4388089ff0 msgr2=0x7f43880cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:18.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 shutdown_connections 2023-10-19T10:37:18.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.484+0000 7f438d7d0640 1 -- 172.21.15.50:0/148444550 wait complete. 2023-10-19T10:37:18.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:18.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.604+0000 7f5d8dda0640 1 Processor -- start 2023-10-19T10:37:18.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.604+0000 7f5d8dda0640 1 -- start start 2023-10-19T10:37:18.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d8dda0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815a3b0 0x7f5d88071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d8dda0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5d8815df00 0x7f5d880b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d8dda0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d880cb520 con 0x7f5d8806f6e0 2023-10-19T10:37:18.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d8dda0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8815eea0 con 0x7f5d8815df00 2023-10-19T10:37:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d8dda0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8815a790 con 0x7f5d8815a3b0 2023-10-19T10:37:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d86ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5d8815df00 0x7f5d880b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d86ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5d8815df00 0x7f5d880b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38776/0 (socket says 172.21.15.50:38776) 2023-10-19T10:37:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d87fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5d8806f6e0 0x7f5d880c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33502/0 (socket says 172.21.15.50:33502) 2023-10-19T10:37:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d86ffd640 1 -- 172.21.15.50:0/3833990334 learned_addr learned my addr 172.21.15.50:0/3833990334 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.605+0000 7f5d877fe640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815a3b0 0x7f5d88071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d86ffd640 1 -- 172.21.15.50:0/3833990334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815a3b0 msgr2=0x7f5d88071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d86ffd640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815a3b0 0x7f5d88071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d86ffd640 1 -- 172.21.15.50:0/3833990334 >> v1:172.21.15.50:6789/0 conn(0x7f5d8806f6e0 legacy=0x7f5d880c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d86ffd640 1 -- 172.21.15.50:0/3833990334 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8815e360 con 0x7f5d8815df00 2023-10-19T10:37:18.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d867fc640 1 -- 172.21.15.50:0/3833990334 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161122574 0 0) 0x7f5d880cb520 con 0x7f5d8806f6e0 2023-10-19T10:37:18.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d877fe640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815a3b0 0x7f5d88071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:18.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d86ffd640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8815df00 0x7f5d880b0ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009870 tx=0x7f5d7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a376c3bf06820713 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d867fc640 1 -- 172.21.15.50:0/3833990334 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c030840 con 0x7f5d8815df00 2023-10-19T10:37:18.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d867fc640 1 -- 172.21.15.50:0/3833990334 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0309a0 con 0x7f5d8815df00 2023-10-19T10:37:18.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.606+0000 7f5d867fc640 1 -- 172.21.15.50:0/3833990334 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c030d30 con 0x7f5d8815df00 2023-10-19T10:37:18.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 -- 172.21.15.50:0/3833990334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8815df00 msgr2=0x7f5d880b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8815df00 0x7f5d880b0ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009870 tx=0x7f5d7c02b980 comp rx=0 tx=0).stop 2023-10-19T10:37:18.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 -- 172.21.15.50:0/3833990334 shutdown_connections 2023-10-19T10:37:18.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8815df00 0x7f5d880b0ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/3833990334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815a3b0 0x7f5d88071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 -- 172.21.15.50:0/3833990334 >> 172.21.15.50:0/3833990334 conn(0x7f5d88069e90 msgr2=0x7f5d880afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:18.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 -- 172.21.15.50:0/3833990334 shutdown_connections 2023-10-19T10:37:18.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.607+0000 7f5d8dda0640 1 -- 172.21.15.50:0/3833990334 wait complete. 2023-10-19T10:37:18.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.608+0000 7f5d8dda0640 1 Processor -- start 2023-10-19T10:37:18.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.608+0000 7f5d8dda0640 1 -- start start 2023-10-19T10:37:18.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d8dda0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 0x7f5d8815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d8dda0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8815a3b0 0x7f5d8814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d8dda0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815df00 0x7f5d8815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d8dda0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8815b0b0 con 0x7f5d8806f6e0 2023-10-19T10:37:18.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d8dda0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8814eb60 con 0x7f5d8815a3b0 2023-10-19T10:37:18.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d8dda0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880c5ab0 con 0x7f5d8815df00 2023-10-19T10:37:18.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 0x7f5d8815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 0x7f5d8815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38788/0 (socket says 172.21.15.50:38788) 2023-10-19T10:37:18.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 -- 172.21.15.50:0/1586969473 learned_addr learned my addr 172.21.15.50:0/1586969473 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:18.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d86ffd640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8815a3b0 0x7f5d8814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d87fff640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815df00 0x7f5d8815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 -- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815df00 msgr2=0x7f5d8815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815df00 0x7f5d8815ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 -- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8815a3b0 msgr2=0x7f5d8814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8815a3b0 0x7f5d8814e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.609+0000 7f5d877fe640 1 -- 172.21.15.50:0/1586969473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0094d0 con 0x7f5d8806f6e0 2023-10-19T10:37:18.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.610+0000 7f5d877fe640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 0x7f5d8815d560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800e7e0 tx=0x7f5d7800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.610+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78004030 con 0x7f5d8806f6e0 2023-10-19T10:37:18.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.610+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d78004190 con 0x7f5d8806f6e0 2023-10-19T10:37:18.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.610+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78004520 con 0x7f5d8806f6e0 2023-10-19T10:37:18.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.610+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8815b2a0 con 0x7f5d8806f6e0 2023-10-19T10:37:18.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.610+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d880c5460 con 0x7f5d8806f6e0 2023-10-19T10:37:18.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.611+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88073990 con 0x7f5d8806f6e0 2023-10-19T10:37:18.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.611+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d7800c6d0 con 0x7f5d8806f6e0 2023-10-19T10:37:18.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.612+0000 7f5d6ffff640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d7406a050 0x7f5d7406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.612+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5d78088540 con 0x7f5d8806f6e0 2023-10-19T10:37:18.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.612+0000 7f5d86ffd640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d7406a050 0x7f5d7406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.612+0000 7f5d86ffd640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d7406a050 0x7f5d7406c510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c036000 tx=0x7f5d7c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.616+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5d78053940 con 0x7f5d8806f6e0 2023-10-19T10:37:18.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.793+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d880b07b0 con 0x7f5d8806f6e0 2023-10-19T10:37:18.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.794+0000 7f5d6ffff640 1 -- 172.21.15.50:0/1586969473 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f5d78056f60 con 0x7f5d8806f6e0 2023-10-19T10:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:18.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.796+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d7406a050 msgr2=0x7f5d7406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.796+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d7406a050 0x7f5d7406c510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c036000 tx=0x7f5d7c03b040 comp rx=0 tx=0).stop 2023-10-19T10:37:18.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.796+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 msgr2=0x7f5d8815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.796+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 0x7f5d8815d560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800e7e0 tx=0x7f5d7800eca0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 shutdown_connections 2023-10-19T10:37:18.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d7406a050 0x7f5d7406c510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d8815df00 0x7f5d8815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8815a3b0 0x7f5d8814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 --2- 172.21.15.50:0/1586969473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d8806f6e0 0x7f5d8815d560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 >> 172.21.15.50:0/1586969473 conn(0x7f5d88069e90 msgr2=0x7f5d880cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:18.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 shutdown_connections 2023-10-19T10:37:18.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.797+0000 7f5d8dda0640 1 -- 172.21.15.50:0/1586969473 wait complete. 2023-10-19T10:37:18.810 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-19T10:37:18.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:37:18.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.914+0000 7fc80b684640 1 Processor -- start 2023-10-19T10:37:18.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.914+0000 7fc80b684640 1 -- start start 2023-10-19T10:37:18.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc80b684640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc80415c810 0x7fc80406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc80b684640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80406c870 con 0x7fc8040cd3a0 2023-10-19T10:37:18.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc80b684640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80406cb50 con 0x7fc80406aa40 2023-10-19T10:37:18.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc80b684640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80406d4f0 con 0x7fc80415c810 2023-10-19T10:37:18.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc809c21640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc8040cd3a0 0x7fc8040b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33518/0 (socket says 172.21.15.50:33518) 2023-10-19T10:37:18.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc809c21640 1 -- 172.21.15.50:0/4131375202 learned_addr learned my addr 172.21.15.50:0/4131375202 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:18.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc809420640 1 --2- 172.21.15.50:0/4131375202 >> v2:172.21.15.183:3300/0 conn(0x7fc80415c810 0x7fc80406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200180230 0 0) 0x7fc80406c870 con 0x7fc8040cd3a0 2023-10-19T10:37:18.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f0003520 con 0x7fc8040cd3a0 2023-10-19T10:37:18.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307268522 0 0) 0x7fc7f0003520 con 0x7fc8040cd3a0 2023-10-19T10:37:18.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 >> v2:172.21.15.183:3300/0 conn(0x7fc80415c810 msgr2=0x7fc80406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:18.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc7fbfff640 1 --2- 172.21.15.50:0/4131375202 >> v2:172.21.15.183:3300/0 conn(0x7fc80415c810 0x7fc80406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.915+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 >> v1:172.21.15.183:6789/0 conn(0x7fc80406aa40 legacy=0x7fc8040ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80406ce30 con 0x7fc8040cd3a0 2023-10-19T10:37:18.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fc7fc0019f0 con 0x7fc8040cd3a0 2023-10-19T10:37:18.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 >> v1:172.21.15.50:6789/0 conn(0x7fc8040cd3a0 legacy=0x7fc8040b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7f0003970 0x7fc7f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f00047c0 0x7fc7f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0005180 0x7fc7f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80406d4f0 con 0x7fc7f0003970 2023-10-19T10:37:18.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000c020 con 0x7fc7f00047c0 2023-10-19T10:37:18.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000c1e0 con 0x7fc7f0005180 2023-10-19T10:37:18.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809420640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7f0003970 0x7fc7f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809c21640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0005180 0x7fc7f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc808c1f640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f00047c0 0x7fc7f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809420640 1 -- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0005180 msgr2=0x7fc7f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:18.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809420640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0005180 0x7fc7f000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809420640 1 -- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f00047c0 msgr2=0x7fc7f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:18.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809420640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f00047c0 0x7fc7f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809420640 1 -- 172.21.15.50:0/4131375202 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f0003520 con 0x7fc7f0003970 2023-10-19T10:37:18.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809c21640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0005180 0x7fc7f000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:18.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc808c1f640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f00047c0 0x7fc7f0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:18.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.916+0000 7fc809c21640 1 -- 172.21.15.50:0/4131375202 reap_dead start 2023-10-19T10:37:18.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc809420640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7f0003970 0x7fc7f0003d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc800009ac0 tx=0x7fc7f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a772c3f62180820 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80002c6b0 con 0x7fc7f0003970 2023-10-19T10:37:18.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80002c810 con 0x7fc7f0003970 2023-10-19T10:37:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc7fbfff640 1 -- 172.21.15.50:0/4131375202 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80002cba0 con 0x7fc7f0003970 2023-10-19T10:37:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 -- 172.21.15.50:0/4131375202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7f0003970 msgr2=0x7fc7f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7f0003970 0x7fc7f0003d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc800009ac0 tx=0x7fc7f00057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 -- 172.21.15.50:0/4131375202 shutdown_connections 2023-10-19T10:37:18.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 --2- 172.21.15.50:0/4131375202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7f0003970 0x7fc7f0003d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 -- 172.21.15.50:0/4131375202 >> 172.21.15.50:0/4131375202 conn(0x7fc8040ce980 msgr2=0x7fc8040ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:18.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 -- 172.21.15.50:0/4131375202 shutdown_connections 2023-10-19T10:37:18.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.917+0000 7fc80b684640 1 -- 172.21.15.50:0/4131375202 wait complete. 2023-10-19T10:37:18.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 Processor -- start 2023-10-19T10:37:18.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 -- start start 2023-10-19T10:37:18.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0003970 0x7fc80414a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8040b42b0 0x7fc8041590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 0x7fc8041efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040a9730 con 0x7fc8041595f0 2023-10-19T10:37:18.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc804159a70 con 0x7fc8040b42b0 2023-10-19T10:37:18.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.919+0000 7fc80b684640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80414a9a0 con 0x7fc7f0003970 2023-10-19T10:37:18.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 0x7fc8041efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 0x7fc8041efbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38806/0 (socket says 172.21.15.50:38806) 2023-10-19T10:37:18.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 -- 172.21.15.50:0/604947549 learned_addr learned my addr 172.21.15.50:0/604947549 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:18.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc808c1f640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8040b42b0 0x7fc8041590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809420640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0003970 0x7fc80414a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 -- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0003970 msgr2=0x7fc80414a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0003970 0x7fc80414a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 -- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8040b42b0 msgr2=0x7fc8041590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:18.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8040b42b0 0x7fc8041590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:18.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 -- 172.21.15.50:0/604947549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8040a8ab0 con 0x7fc8041595f0 2023-10-19T10:37:18.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc809c21640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 0x7fc8041efbc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc001f30 tx=0x7fc7fc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00eb90 con 0x7fc8041595f0 2023-10-19T10:37:18.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00ecf0 con 0x7fc8041595f0 2023-10-19T10:37:18.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc013660 con 0x7fc8041595f0 2023-10-19T10:37:18.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8040a8d90 con 0x7fc8041595f0 2023-10-19T10:37:18.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.920+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8040a9350 con 0x7fc8041595f0 2023-10-19T10:37:18.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.921+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8040a94e0 con 0x7fc8041595f0 2023-10-19T10:37:18.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.922+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc7fc0137c0 con 0x7fc8041595f0 2023-10-19T10:37:18.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.922+0000 7fc7f9ffb640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc7e806a050 0x7fc7e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:18.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.922+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc7fc088a90 con 0x7fc8041595f0 2023-10-19T10:37:18.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.922+0000 7fc809420640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc7e806a050 0x7fc7e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:18.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.923+0000 7fc809420640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc7e806a050 0x7fc7e806c510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc800036000 tx=0x7fc800030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:18.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:18.926+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc7fc053f40 con 0x7fc8041595f0 2023-10-19T10:37:19.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.099+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc80406d110 con 0x7fc7e806a050 2023-10-19T10:37:19.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.111+0000 7fc7f9ffb640 1 -- 172.21.15.50:0/604947549 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367389 (secure 0 0 0) 0x7fc800004040 con 0x7fc7e806a050 2023-10-19T10:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:37:19.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc7e806a050 msgr2=0x7fc7e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc7e806a050 0x7fc7e806c510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc800036000 tx=0x7fc800030000 comp rx=0 tx=0).stop 2023-10-19T10:37:19.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 msgr2=0x7fc8041efbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 0x7fc8041efbc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc001f30 tx=0x7fc7fc00e530 comp rx=0 tx=0).stop 2023-10-19T10:37:19.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 shutdown_connections 2023-10-19T10:37:19.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc7e806a050 0x7fc7e806c510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8041595f0 0x7fc8041efbc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8040b42b0 0x7fc8041590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 --2- 172.21.15.50:0/604947549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7f0003970 0x7fc80414a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 >> 172.21.15.50:0/604947549 conn(0x7fc8040ce980 msgr2=0x7fc8040700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:19.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 shutdown_connections 2023-10-19T10:37:19.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.115+0000 7fc80b684640 1 -- 172.21.15.50:0/604947549 wait complete. 2023-10-19T10:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-19T10:37:18.468770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236080,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738640,"statfs":{"total":773094113280,"available":772852367360,"internally_reserved":0,"allocated":3579904,"data_stored":2146982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003436"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096693+0000","last_change":"2023-10-19T10:36:59.359352+0000","last_active":"2023-10-19T10:37:00.096693+0000","last_peered":"2023-10-19T10:37:00.096693+0000","last_clean":"2023-10-19T10:37:00.096693+0000","last_became_active":"2023-10-19T10:36:59.358992+0000","last_became_peered":"2023-10-19T10:36:59.358992+0000","last_unstale":"2023-10-19T10:37:00.096693+0000","last_undegraded":"2023-10-19T10:37:00.096693+0000","last_fullsized":"2023-10-19T10:37:00.096693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:36:39.869930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278383+0000","last_change":"2023-10-19T10:36:54.305740+0000","last_active":"2023-10-19T10:37:00.278383+0000","last_peered":"2023-10-19T10:37:00.278383+0000","last_clean":"2023-10-19T10:37:00.278383+0000","last_became_active":"2023-10-19T10:36:54.305192+0000","last_became_peered":"2023-10-19T10:36:54.305192+0000","last_unstale":"2023-10-19T10:37:00.278383+0000","last_undegraded":"2023-10-19T10:37:00.278383+0000","last_fullsized":"2023-10-19T10:37:00.278383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097666+0000","last_change":"2023-10-19T10:36:55.108193+0000","last_active":"2023-10-19T10:37:00.097666+0000","last_peered":"2023-10-19T10:37:00.097666+0000","last_clean":"2023-10-19T10:37:00.097666+0000","last_became_active":"2023-10-19T10:36:55.107375+0000","last_became_peered":"2023-10-19T10:36:55.107375+0000","last_unstale":"2023-10-19T10:37:00.097666+0000","last_undegraded":"2023-10-19T10:37:00.097666+0000","last_fullsized":"2023-10-19T10:37:00.097666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:13:03.441540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097567+0000","last_change":"2023-10-19T10:36:59.344962+0000","last_active":"2023-10-19T10:37:00.097567+0000","last_peered":"2023-10-19T10:37:00.097567+0000","last_clean":"2023-10-19T10:37:00.097567+0000","last_became_active":"2023-10-19T10:36:59.344519+0000","last_became_peered":"2023-10-19T10:36:59.344519+0000","last_unstale":"2023-10-19T10:37:00.097567+0000","last_undegraded":"2023-10-19T10:37:00.097567+0000","last_fullsized":"2023-10-19T10:37:00.097567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:33.660694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:54.310896+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:54.310583+0000","last_became_peered":"2023-10-19T10:36:54.310583+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:56:34.283482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323668+0000","last_change":"2023-10-19T10:36:55.104115+0000","last_active":"2023-10-19T10:37:00.323668+0000","last_peered":"2023-10-19T10:37:00.323668+0000","last_clean":"2023-10-19T10:37:00.323668+0000","last_became_active":"2023-10-19T10:36:55.103785+0000","last_became_peered":"2023-10-19T10:36:55.103785+0000","last_unstale":"2023-10-19T10:37:00.323668+0000","last_undegraded":"2023-10-19T10:37:00.323668+0000","last_fullsized":"2023-10-19T10:37:00.323668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324072+0000","last_change":"2023-10-19T10:36:59.350493+0000","last_active":"2023-10-19T10:37:00.324072+0000","last_peered":"2023-10-19T10:37:00.324072+0000","last_clean":"2023-10-19T10:37:00.324072+0000","last_became_active":"2023-10-19T10:36:59.350100+0000","last_became_peered":"2023-10-19T10:36:59.350100+0000","last_unstale":"2023-10-19T10:37:00.324072+0000","last_undegraded":"2023-10-19T10:37:00.324072+0000","last_fullsized":"2023-10-19T10:37:00.324072+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324355+0000","last_change":"2023-10-19T10:36:54.307087+0000","last_active":"2023-10-19T10:37:00.324355+0000","last_peered":"2023-10-19T10:37:00.324355+0000","last_clean":"2023-10-19T10:37:00.324355+0000","last_became_active":"2023-10-19T10:36:54.306818+0000","last_became_peered":"2023-10-19T10:36:54.306818+0000","last_unstale":"2023-10-19T10:37:00.324355+0000","last_undegraded":"2023-10-19T10:37:00.324355+0000","last_fullsized":"2023-10-19T10:37:00.324355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:25.504656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098032+0000","last_change":"2023-10-19T10:36:55.111676+0000","last_active":"2023-10-19T10:37:00.098032+0000","last_peered":"2023-10-19T10:37:00.098032+0000","last_clean":"2023-10-19T10:37:00.098032+0000","last_became_active":"2023-10-19T10:36:55.110442+0000","last_became_peered":"2023-10-19T10:36:55.110442+0000","last_unstale":"2023-10-19T10:37:00.098032+0000","last_undegraded":"2023-10-19T10:37:00.098032+0000","last_fullsized":"2023-10-19T10:37:00.098032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:51:02.897898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097851+0000","last_change":"2023-10-19T10:36:59.357016+0000","last_active":"2023-10-19T10:37:00.097851+0000","last_peered":"2023-10-19T10:37:00.097851+0000","last_clean":"2023-10-19T10:37:00.097851+0000","last_became_active":"2023-10-19T10:36:59.356666+0000","last_became_peered":"2023-10-19T10:36:59.356666+0000","last_unstale":"2023-10-19T10:37:00.097851+0000","last_undegraded":"2023-10-19T10:37:00.097851+0000","last_fullsized":"2023-10-19T10:37:00.097851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324054+0000","last_change":"2023-10-19T10:36:54.306526+0000","last_active":"2023-10-19T10:37:00.324054+0000","last_peered":"2023-10-19T10:37:00.324054+0000","last_clean":"2023-10-19T10:37:00.324054+0000","last_became_active":"2023-10-19T10:36:54.306285+0000","last_became_peered":"2023-10-19T10:36:54.306285+0000","last_unstale":"2023-10-19T10:37:00.324054+0000","last_undegraded":"2023-10-19T10:37:00.324054+0000","last_fullsized":"2023-10-19T10:37:00.324054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:55.717306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098946+0000","last_change":"2023-10-19T10:36:55.115029+0000","last_active":"2023-10-19T10:37:00.098946+0000","last_peered":"2023-10-19T10:37:00.098946+0000","last_clean":"2023-10-19T10:37:00.098946+0000","last_became_active":"2023-10-19T10:36:55.114754+0000","last_became_peered":"2023-10-19T10:36:55.114754+0000","last_unstale":"2023-10-19T10:37:00.098946+0000","last_undegraded":"2023-10-19T10:37:00.098946+0000","last_fullsized":"2023-10-19T10:37:00.098946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:30:31.789571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097648+0000","last_change":"2023-10-19T10:36:59.355687+0000","last_active":"2023-10-19T10:37:00.097648+0000","last_peered":"2023-10-19T10:37:00.097648+0000","last_clean":"2023-10-19T10:37:00.097648+0000","last_became_active":"2023-10-19T10:36:59.355294+0000","last_became_peered":"2023-10-19T10:36:59.355294+0000","last_unstale":"2023-10-19T10:37:00.097648+0000","last_undegraded":"2023-10-19T10:37:00.097648+0000","last_fullsized":"2023-10-19T10:37:00.097648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:39:00.759461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278352+0000","last_change":"2023-10-19T10:36:54.307829+0000","last_active":"2023-10-19T10:37:00.278352+0000","last_peered":"2023-10-19T10:37:00.278352+0000","last_clean":"2023-10-19T10:37:00.278352+0000","last_became_active":"2023-10-19T10:36:54.307571+0000","last_became_peered":"2023-10-19T10:36:54.307571+0000","last_unstale":"2023-10-19T10:37:00.278352+0000","last_undegraded":"2023-10-19T10:37:00.278352+0000","last_fullsized":"2023-10-19T10:37:00.278352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:45.887676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323888+0000","last_change":"2023-10-19T10:36:55.112212+0000","last_active":"2023-10-19T10:37:00.323888+0000","last_peered":"2023-10-19T10:37:00.323888+0000","last_clean":"2023-10-19T10:37:00.323888+0000","last_became_active":"2023-10-19T10:36:55.111854+0000","last_became_peered":"2023-10-19T10:36:55.111854+0000","last_unstale":"2023-10-19T10:37:00.323888+0000","last_undegraded":"2023-10-19T10:37:00.323888+0000","last_fullsized":"2023-10-19T10:37:00.323888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:49:46.713742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099140+0000","last_change":"2023-10-19T10:36:59.352206+0000","last_active":"2023-10-19T10:37:00.099140+0000","last_peered":"2023-10-19T10:37:00.099140+0000","last_clean":"2023-10-19T10:37:00.099140+0000","last_became_active":"2023-10-19T10:36:59.351813+0000","last_became_peered":"2023-10-19T10:36:59.351813+0000","last_unstale":"2023-10-19T10:37:00.099140+0000","last_undegraded":"2023-10-19T10:37:00.099140+0000","last_fullsized":"2023-10-19T10:37:00.099140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:02.738858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098273+0000","last_change":"2023-10-19T10:36:54.305390+0000","last_active":"2023-10-19T10:37:00.098273+0000","last_peered":"2023-10-19T10:37:00.098273+0000","last_clean":"2023-10-19T10:37:00.098273+0000","last_became_active":"2023-10-19T10:36:54.304960+0000","last_became_peered":"2023-10-19T10:36:54.304960+0000","last_unstale":"2023-10-19T10:37:00.098273+0000","last_undegraded":"2023-10-19T10:37:00.098273+0000","last_fullsized":"2023-10-19T10:37:00.098273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:21.713184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324108+0000","last_change":"2023-10-19T10:36:55.114266+0000","last_active":"2023-10-19T10:37:00.324108+0000","last_peered":"2023-10-19T10:37:00.324108+0000","last_clean":"2023-10-19T10:37:00.324108+0000","last_became_active":"2023-10-19T10:36:55.113992+0000","last_became_peered":"2023-10-19T10:36:55.113992+0000","last_unstale":"2023-10-19T10:37:00.324108+0000","last_undegraded":"2023-10-19T10:37:00.324108+0000","last_fullsized":"2023-10-19T10:37:00.324108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:17.080140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323688+0000","last_change":"2023-10-19T10:36:59.349278+0000","last_active":"2023-10-19T10:37:00.323688+0000","last_peered":"2023-10-19T10:37:00.323688+0000","last_clean":"2023-10-19T10:37:00.323688+0000","last_became_active":"2023-10-19T10:36:59.348852+0000","last_became_peered":"2023-10-19T10:36:59.348852+0000","last_unstale":"2023-10-19T10:37:00.323688+0000","last_undegraded":"2023-10-19T10:37:00.323688+0000","last_fullsized":"2023-10-19T10:37:00.323688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:20.365692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096958+0000","last_change":"2023-10-19T10:36:54.306039+0000","last_active":"2023-10-19T10:37:00.096958+0000","last_peered":"2023-10-19T10:37:00.096958+0000","last_clean":"2023-10-19T10:37:00.096958+0000","last_became_active":"2023-10-19T10:36:54.304502+0000","last_became_peered":"2023-10-19T10:36:54.304502+0000","last_unstale":"2023-10-19T10:37:00.096958+0000","last_undegraded":"2023-10-19T10:37:00.096958+0000","last_fullsized":"2023-10-19T10:37:00.096958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:07:52.900648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278255+0000","last_change":"2023-10-19T10:36:55.114034+0000","last_active":"2023-10-19T10:37:00.278255+0000","last_peered":"2023-10-19T10:37:00.278255+0000","last_clean":"2023-10-19T10:37:00.278255+0000","last_became_active":"2023-10-19T10:36:55.113734+0000","last_became_peered":"2023-10-19T10:36:55.113734+0000","last_unstale":"2023-10-19T10:37:00.278255+0000","last_undegraded":"2023-10-19T10:37:00.278255+0000","last_fullsized":"2023-10-19T10:37:00.278255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278113+0000","last_change":"2023-10-19T10:36:59.349586+0000","last_active":"2023-10-19T10:37:00.278113+0000","last_peered":"2023-10-19T10:37:00.278113+0000","last_clean":"2023-10-19T10:37:00.278113+0000","last_became_active":"2023-10-19T10:36:59.349176+0000","last_became_peered":"2023-10-19T10:36:59.349176+0000","last_unstale":"2023-10-19T10:37:00.278113+0000","last_undegraded":"2023-10-19T10:37:00.278113+0000","last_fullsized":"2023-10-19T10:37:00.278113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323262+0000","last_change":"2023-10-19T10:36:54.310735+0000","last_active":"2023-10-19T10:37:00.323262+0000","last_peered":"2023-10-19T10:37:00.323262+0000","last_clean":"2023-10-19T10:37:00.323262+0000","last_became_active":"2023-10-19T10:36:54.309734+0000","last_became_peered":"2023-10-19T10:36:54.309734+0000","last_unstale":"2023-10-19T10:37:00.323262+0000","last_undegraded":"2023-10-19T10:37:00.323262+0000","last_fullsized":"2023-10-19T10:37:00.323262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323712+0000","last_change":"2023-10-19T10:36:55.113078+0000","last_active":"2023-10-19T10:37:00.323712+0000","last_peered":"2023-10-19T10:37:00.323712+0000","last_clean":"2023-10-19T10:37:00.323712+0000","last_became_active":"2023-10-19T10:36:55.112775+0000","last_became_peered":"2023-10-19T10:36:55.112775+0000","last_unstale":"2023-10-19T10:37:00.323712+0000","last_undegraded":"2023-10-19T10:37:00.323712+0000","last_fullsized":"2023-10-19T10:37:00.323712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:12:41.211214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322933+0000","last_change":"2023-10-19T10:36:55.105391+0000","last_active":"2023-10-19T10:37:00.322933+0000","last_peered":"2023-10-19T10:37:00.322933+0000","last_clean":"2023-10-19T10:37:00.322933+0000","last_became_active":"2023-10-19T10:36:55.105112+0000","last_became_peered":"2023-10-19T10:36:55.105112+0000","last_unstale":"2023-10-19T10:37:00.322933+0000","last_undegraded":"2023-10-19T10:37:00.322933+0000","last_fullsized":"2023-10-19T10:37:00.322933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278476+0000","last_change":"2023-10-19T10:36:54.305435+0000","last_active":"2023-10-19T10:37:00.278476+0000","last_peered":"2023-10-19T10:37:00.278476+0000","last_clean":"2023-10-19T10:37:00.278476+0000","last_became_active":"2023-10-19T10:36:54.305123+0000","last_became_peered":"2023-10-19T10:36:54.305123+0000","last_unstale":"2023-10-19T10:37:00.278476+0000","last_undegraded":"2023-10-19T10:37:00.278476+0000","last_fullsized":"2023-10-19T10:37:00.278476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323209+0000","last_change":"2023-10-19T10:36:59.350259+0000","last_active":"2023-10-19T10:37:00.323209+0000","last_peered":"2023-10-19T10:37:00.323209+0000","last_clean":"2023-10-19T10:37:00.323209+0000","last_became_active":"2023-10-19T10:36:59.349878+0000","last_became_peered":"2023-10-19T10:36:59.349878+0000","last_unstale":"2023-10-19T10:37:00.323209+0000","last_undegraded":"2023-10-19T10:37:00.323209+0000","last_fullsized":"2023-10-19T10:37:00.323209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323250+0000","last_change":"2023-10-19T10:36:55.113112+0000","last_active":"2023-10-19T10:37:00.323250+0000","last_peered":"2023-10-19T10:37:00.323250+0000","last_clean":"2023-10-19T10:37:00.323250+0000","last_became_active":"2023-10-19T10:36:55.110232+0000","last_became_peered":"2023-10-19T10:36:55.110232+0000","last_unstale":"2023-10-19T10:37:00.323250+0000","last_undegraded":"2023-10-19T10:37:00.323250+0000","last_fullsized":"2023-10-19T10:37:00.323250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324722+0000","last_change":"2023-10-19T10:36:54.310064+0000","last_active":"2023-10-19T10:37:00.324722+0000","last_peered":"2023-10-19T10:37:00.324722+0000","last_clean":"2023-10-19T10:37:00.324722+0000","last_became_active":"2023-10-19T10:36:54.309581+0000","last_became_peered":"2023-10-19T10:36:54.309581+0000","last_unstale":"2023-10-19T10:37:00.324722+0000","last_undegraded":"2023-10-19T10:37:00.324722+0000","last_fullsized":"2023-10-19T10:37:00.324722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:16.076243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098023+0000","last_change":"2023-10-19T10:36:59.355292+0000","last_active":"2023-10-19T10:37:00.098023+0000","last_peered":"2023-10-19T10:37:00.098023+0000","last_clean":"2023-10-19T10:37:00.098023+0000","last_became_active":"2023-10-19T10:36:59.354866+0000","last_became_peered":"2023-10-19T10:36:59.354866+0000","last_unstale":"2023-10-19T10:37:00.098023+0000","last_undegraded":"2023-10-19T10:37:00.098023+0000","last_fullsized":"2023-10-19T10:37:00.098023+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098814+0000","last_change":"2023-10-19T10:36:54.310722+0000","last_active":"2023-10-19T10:37:00.098814+0000","last_peered":"2023-10-19T10:37:00.098814+0000","last_clean":"2023-10-19T10:37:00.098814+0000","last_became_active":"2023-10-19T10:36:54.310410+0000","last_became_peered":"2023-10-19T10:36:54.310410+0000","last_unstale":"2023-10-19T10:37:00.098814+0000","last_undegraded":"2023-10-19T10:37:00.098814+0000","last_fullsized":"2023-10-19T10:37:00.098814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:00.221492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097163+0000","last_change":"2023-10-19T10:36:55.110762+0000","last_active":"2023-10-19T10:37:00.097163+0000","last_peered":"2023-10-19T10:37:00.097163+0000","last_clean":"2023-10-19T10:37:00.097163+0000","last_became_active":"2023-10-19T10:36:55.110503+0000","last_became_peered":"2023-10-19T10:36:55.110503+0000","last_unstale":"2023-10-19T10:37:00.097163+0000","last_undegraded":"2023-10-19T10:37:00.097163+0000","last_fullsized":"2023-10-19T10:37:00.097163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:14.103808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323380+0000","last_change":"2023-10-19T10:36:59.352779+0000","last_active":"2023-10-19T10:37:00.323380+0000","last_peered":"2023-10-19T10:37:00.323380+0000","last_clean":"2023-10-19T10:37:00.323380+0000","last_became_active":"2023-10-19T10:36:59.352362+0000","last_became_peered":"2023-10-19T10:36:59.352362+0000","last_unstale":"2023-10-19T10:37:00.323380+0000","last_undegraded":"2023-10-19T10:37:00.323380+0000","last_fullsized":"2023-10-19T10:37:00.323380+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:53.134651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:04.746420+0000","last_change":"2023-10-19T10:36:54.310368+0000","last_active":"2023-10-19T10:37:04.746420+0000","last_peered":"2023-10-19T10:37:04.746420+0000","last_clean":"2023-10-19T10:37:04.746420+0000","last_became_active":"2023-10-19T10:36:54.309745+0000","last_became_peered":"2023-10-19T10:36:54.309745+0000","last_unstale":"2023-10-19T10:37:04.746420+0000","last_undegraded":"2023-10-19T10:37:04.746420+0000","last_fullsized":"2023-10-19T10:37:04.746420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:15:27.898150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099416+0000","last_change":"2023-10-19T10:36:55.106300+0000","last_active":"2023-10-19T10:37:00.099416+0000","last_peered":"2023-10-19T10:37:00.099416+0000","last_clean":"2023-10-19T10:37:00.099416+0000","last_became_active":"2023-10-19T10:36:55.106045+0000","last_became_peered":"2023-10-19T10:36:55.106045+0000","last_unstale":"2023-10-19T10:37:00.099416+0000","last_undegraded":"2023-10-19T10:37:00.099416+0000","last_fullsized":"2023-10-19T10:37:00.099416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:28:29.050287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096752+0000","last_change":"2023-10-19T10:36:59.358801+0000","last_active":"2023-10-19T10:37:00.096752+0000","last_peered":"2023-10-19T10:37:00.096752+0000","last_clean":"2023-10-19T10:37:00.096752+0000","last_became_active":"2023-10-19T10:36:59.356906+0000","last_became_peered":"2023-10-19T10:36:59.356906+0000","last_unstale":"2023-10-19T10:37:00.096752+0000","last_undegraded":"2023-10-19T10:37:00.096752+0000","last_fullsized":"2023-10-19T10:37:00.096752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:56.099256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097964+0000","last_change":"2023-10-19T10:36:55.105397+0000","last_active":"2023-10-19T10:37:00.097964+0000","last_peered":"2023-10-19T10:37:00.097964+0000","last_clean":"2023-10-19T10:37:00.097964+0000","last_became_active":"2023-10-19T10:36:55.105016+0000","last_became_peered":"2023-10-19T10:36:55.105016+0000","last_unstale":"2023-10-19T10:37:00.097964+0000","last_undegraded":"2023-10-19T10:37:00.097964+0000","last_fullsized":"2023-10-19T10:37:00.097964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:13.604819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323795+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:37:00.323795+0000","last_peered":"2023-10-19T10:37:00.323795+0000","last_clean":"2023-10-19T10:37:00.323795+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:37:00.323795+0000","last_undegraded":"2023-10-19T10:37:00.323795+0000","last_fullsized":"2023-10-19T10:37:00.323795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:47:38.405947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278495+0000","last_change":"2023-10-19T10:36:54.301501+0000","last_active":"2023-10-19T10:37:00.278495+0000","last_peered":"2023-10-19T10:37:00.278495+0000","last_clean":"2023-10-19T10:37:00.278495+0000","last_became_active":"2023-10-19T10:36:54.301193+0000","last_became_peered":"2023-10-19T10:36:54.301193+0000","last_unstale":"2023-10-19T10:37:00.278495+0000","last_undegraded":"2023-10-19T10:37:00.278495+0000","last_fullsized":"2023-10-19T10:37:00.278495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323709+0000","last_change":"2023-10-19T10:36:59.351528+0000","last_active":"2023-10-19T10:37:00.323709+0000","last_peered":"2023-10-19T10:37:00.323709+0000","last_clean":"2023-10-19T10:37:00.323709+0000","last_became_active":"2023-10-19T10:36:59.351102+0000","last_became_peered":"2023-10-19T10:36:59.351102+0000","last_unstale":"2023-10-19T10:37:00.323709+0000","last_undegraded":"2023-10-19T10:37:00.323709+0000","last_fullsized":"2023-10-19T10:37:00.323709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323080+0000","last_change":"2023-10-19T10:36:55.108015+0000","last_active":"2023-10-19T10:37:00.323080+0000","last_peered":"2023-10-19T10:37:00.323080+0000","last_clean":"2023-10-19T10:37:00.323080+0000","last_became_active":"2023-10-19T10:36:55.107693+0000","last_became_peered":"2023-10-19T10:36:55.107693+0000","last_unstale":"2023-10-19T10:37:00.323080+0000","last_undegraded":"2023-10-19T10:37:00.323080+0000","last_fullsized":"2023-10-19T10:37:00.323080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097888+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:37:00.097888+0000","last_peered":"2023-10-19T10:37:00.097888+0000","last_clean":"2023-10-19T10:37:00.097888+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:37:00.097888+0000","last_undegraded":"2023-10-19T10:37:00.097888+0000","last_fullsized":"2023-10-19T10:37:00.097888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:31:56.734648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324508+0000","last_change":"2023-10-19T10:36:54.310814+0000","last_active":"2023-10-19T10:37:00.324508+0000","last_peered":"2023-10-19T10:37:00.324508+0000","last_clean":"2023-10-19T10:37:00.324508+0000","last_became_active":"2023-10-19T10:36:54.310517+0000","last_became_peered":"2023-10-19T10:36:54.310517+0000","last_unstale":"2023-10-19T10:37:00.324508+0000","last_undegraded":"2023-10-19T10:37:00.324508+0000","last_fullsized":"2023-10-19T10:37:00.324508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:37.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097777+0000","last_change":"2023-10-19T10:36:59.353687+0000","last_active":"2023-10-19T10:37:00.097777+0000","last_peered":"2023-10-19T10:37:00.097777+0000","last_clean":"2023-10-19T10:37:00.097777+0000","last_became_active":"2023-10-19T10:36:59.353297+0000","last_became_peered":"2023-10-19T10:36:59.353297+0000","last_unstale":"2023-10-19T10:37:00.097777+0000","last_undegraded":"2023-10-19T10:37:00.097777+0000","last_fullsized":"2023-10-19T10:37:00.097777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:05:36.096009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278152+0000","last_change":"2023-10-19T10:36:55.118741+0000","last_active":"2023-10-19T10:37:00.278152+0000","last_peered":"2023-10-19T10:37:00.278152+0000","last_clean":"2023-10-19T10:37:00.278152+0000","last_became_active":"2023-10-19T10:36:55.116465+0000","last_became_peered":"2023-10-19T10:36:55.116465+0000","last_unstale":"2023-10-19T10:37:00.278152+0000","last_undegraded":"2023-10-19T10:37:00.278152+0000","last_fullsized":"2023-10-19T10:37:00.278152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099045+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:37:00.099045+0000","last_peered":"2023-10-19T10:37:00.099045+0000","last_clean":"2023-10-19T10:37:00.099045+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:37:00.099045+0000","last_undegraded":"2023-10-19T10:37:00.099045+0000","last_fullsized":"2023-10-19T10:37:00.099045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:12.307342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097428+0000","last_change":"2023-10-19T10:36:54.306800+0000","last_active":"2023-10-19T10:37:00.097428+0000","last_peered":"2023-10-19T10:37:00.097428+0000","last_clean":"2023-10-19T10:37:00.097428+0000","last_became_active":"2023-10-19T10:36:54.306507+0000","last_became_peered":"2023-10-19T10:36:54.306507+0000","last_unstale":"2023-10-19T10:37:00.097428+0000","last_undegraded":"2023-10-19T10:37:00.097428+0000","last_fullsized":"2023-10-19T10:37:00.097428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:41.200123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323296+0000","last_change":"2023-10-19T10:36:59.352359+0000","last_active":"2023-10-19T10:37:00.323296+0000","last_peered":"2023-10-19T10:37:00.323296+0000","last_clean":"2023-10-19T10:37:00.323296+0000","last_became_active":"2023-10-19T10:36:59.351975+0000","last_became_peered":"2023-10-19T10:36:59.351975+0000","last_unstale":"2023-10-19T10:37:00.323296+0000","last_undegraded":"2023-10-19T10:37:00.323296+0000","last_fullsized":"2023-10-19T10:37:00.323296+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:39:36.789471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097181+0000","last_change":"2023-10-19T10:36:55.109753+0000","last_active":"2023-10-19T10:37:00.097181+0000","last_peered":"2023-10-19T10:37:00.097181+0000","last_clean":"2023-10-19T10:37:00.097181+0000","last_became_active":"2023-10-19T10:36:55.109424+0000","last_became_peered":"2023-10-19T10:36:55.109424+0000","last_unstale":"2023-10-19T10:37:00.097181+0000","last_undegraded":"2023-10-19T10:37:00.097181+0000","last_fullsized":"2023-10-19T10:37:00.097181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097504+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:37:00.097504+0000","last_peered":"2023-10-19T10:37:00.097504+0000","last_clean":"2023-10-19T10:37:00.097504+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:37:00.097504+0000","last_undegraded":"2023-10-19T10:37:00.097504+0000","last_fullsized":"2023-10-19T10:37:00.097504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:43.871421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324473+0000","last_change":"2023-10-19T10:36:54.305974+0000","last_active":"2023-10-19T10:37:00.324473+0000","last_peered":"2023-10-19T10:37:00.324473+0000","last_clean":"2023-10-19T10:37:00.324473+0000","last_became_active":"2023-10-19T10:36:54.305710+0000","last_became_peered":"2023-10-19T10:36:54.305710+0000","last_unstale":"2023-10-19T10:37:00.324473+0000","last_undegraded":"2023-10-19T10:37:00.324473+0000","last_fullsized":"2023-10-19T10:37:00.324473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277916+0000","last_change":"2023-10-19T10:36:59.353503+0000","last_active":"2023-10-19T10:37:00.277916+0000","last_peered":"2023-10-19T10:37:00.277916+0000","last_clean":"2023-10-19T10:37:00.277916+0000","last_became_active":"2023-10-19T10:36:59.352985+0000","last_became_peered":"2023-10-19T10:36:59.352985+0000","last_unstale":"2023-10-19T10:37:00.277916+0000","last_undegraded":"2023-10-19T10:37:00.277916+0000","last_fullsized":"2023-10-19T10:37:00.277916+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097415+0000","last_change":"2023-10-19T10:36:55.112577+0000","last_active":"2023-10-19T10:37:00.097415+0000","last_peered":"2023-10-19T10:37:00.097415+0000","last_clean":"2023-10-19T10:37:00.097415+0000","last_became_active":"2023-10-19T10:36:55.112302+0000","last_became_peered":"2023-10-19T10:36:55.112302+0000","last_unstale":"2023-10-19T10:37:00.097415+0000","last_undegraded":"2023-10-19T10:37:00.097415+0000","last_fullsized":"2023-10-19T10:37:00.097415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099330+0000","last_change":"2023-10-19T10:36:54.302990+0000","last_active":"2023-10-19T10:37:00.099330+0000","last_peered":"2023-10-19T10:37:00.099330+0000","last_clean":"2023-10-19T10:37:00.099330+0000","last_became_active":"2023-10-19T10:36:54.302553+0000","last_became_peered":"2023-10-19T10:36:54.302553+0000","last_unstale":"2023-10-19T10:37:00.099330+0000","last_undegraded":"2023-10-19T10:37:00.099330+0000","last_fullsized":"2023-10-19T10:37:00.099330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:07:06.761773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323861+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:37:00.323861+0000","last_peered":"2023-10-19T10:37:00.323861+0000","last_clean":"2023-10-19T10:37:00.323861+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:37:00.323861+0000","last_undegraded":"2023-10-19T10:37:00.323861+0000","last_fullsized":"2023-10-19T10:37:00.323861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:37.072096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277993+0000","last_change":"2023-10-19T10:36:59.351199+0000","last_active":"2023-10-19T10:37:00.277993+0000","last_peered":"2023-10-19T10:37:00.277993+0000","last_clean":"2023-10-19T10:37:00.277993+0000","last_became_active":"2023-10-19T10:36:59.350722+0000","last_became_peered":"2023-10-19T10:36:59.350722+0000","last_unstale":"2023-10-19T10:37:00.277993+0000","last_undegraded":"2023-10-19T10:37:00.277993+0000","last_fullsized":"2023-10-19T10:37:00.277993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097884+0000","last_change":"2023-10-19T10:36:55.108041+0000","last_active":"2023-10-19T10:37:00.097884+0000","last_peered":"2023-10-19T10:37:00.097884+0000","last_clean":"2023-10-19T10:37:00.097884+0000","last_became_active":"2023-10-19T10:36:55.107747+0000","last_became_peered":"2023-10-19T10:36:55.107747+0000","last_unstale":"2023-10-19T10:37:00.097884+0000","last_undegraded":"2023-10-19T10:37:00.097884+0000","last_fullsized":"2023-10-19T10:37:00.097884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096738+0000","last_change":"2023-10-19T10:36:54.307533+0000","last_active":"2023-10-19T10:37:00.096738+0000","last_peered":"2023-10-19T10:37:00.096738+0000","last_clean":"2023-10-19T10:37:00.096738+0000","last_became_active":"2023-10-19T10:36:54.307237+0000","last_became_peered":"2023-10-19T10:36:54.307237+0000","last_unstale":"2023-10-19T10:37:00.096738+0000","last_undegraded":"2023-10-19T10:37:00.096738+0000","last_fullsized":"2023-10-19T10:37:00.096738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:57.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322887+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:37:00.322887+0000","last_peered":"2023-10-19T10:37:00.322887+0000","last_clean":"2023-10-19T10:37:00.322887+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:37:00.322887+0000","last_undegraded":"2023-10-19T10:37:00.322887+0000","last_fullsized":"2023-10-19T10:37:00.322887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:35:30.635889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324367+0000","last_change":"2023-10-19T10:36:59.349089+0000","last_active":"2023-10-19T10:37:00.324367+0000","last_peered":"2023-10-19T10:37:00.324367+0000","last_clean":"2023-10-19T10:37:00.324367+0000","last_became_active":"2023-10-19T10:36:59.348703+0000","last_became_peered":"2023-10-19T10:36:59.348703+0000","last_unstale":"2023-10-19T10:37:00.324367+0000","last_undegraded":"2023-10-19T10:37:00.324367+0000","last_fullsized":"2023-10-19T10:37:00.324367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278271+0000","last_change":"2023-10-19T10:36:55.112250+0000","last_active":"2023-10-19T10:37:00.278271+0000","last_peered":"2023-10-19T10:37:00.278271+0000","last_clean":"2023-10-19T10:37:00.278271+0000","last_became_active":"2023-10-19T10:36:55.111996+0000","last_became_peered":"2023-10-19T10:36:55.111996+0000","last_unstale":"2023-10-19T10:37:00.278271+0000","last_undegraded":"2023-10-19T10:37:00.278271+0000","last_fullsized":"2023-10-19T10:37:00.278271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277550+0000","last_change":"2023-10-19T10:36:54.307098+0000","last_active":"2023-10-19T10:37:00.277550+0000","last_peered":"2023-10-19T10:37:00.277550+0000","last_clean":"2023-10-19T10:37:00.277550+0000","last_became_active":"2023-10-19T10:36:54.306769+0000","last_became_peered":"2023-10-19T10:36:54.306769+0000","last_unstale":"2023-10-19T10:37:00.277550+0000","last_undegraded":"2023-10-19T10:37:00.277550+0000","last_fullsized":"2023-10-19T10:37:00.277550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:13.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098430+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:37:00.098430+0000","last_peered":"2023-10-19T10:37:00.098430+0000","last_clean":"2023-10-19T10:37:00.098430+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:37:00.098430+0000","last_undegraded":"2023-10-19T10:37:00.098430+0000","last_fullsized":"2023-10-19T10:37:00.098430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324610+0000","last_change":"2023-10-19T10:36:59.357456+0000","last_active":"2023-10-19T10:37:00.324610+0000","last_peered":"2023-10-19T10:37:00.324610+0000","last_clean":"2023-10-19T10:37:00.324610+0000","last_became_active":"2023-10-19T10:36:59.357079+0000","last_became_peered":"2023-10-19T10:36:59.357079+0000","last_unstale":"2023-10-19T10:37:00.324610+0000","last_undegraded":"2023-10-19T10:37:00.324610+0000","last_fullsized":"2023-10-19T10:37:00.324610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:52.409495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323740+0000","last_change":"2023-10-19T10:36:55.113749+0000","last_active":"2023-10-19T10:37:00.323740+0000","last_peered":"2023-10-19T10:37:00.323740+0000","last_clean":"2023-10-19T10:37:00.323740+0000","last_became_active":"2023-10-19T10:36:55.113459+0000","last_became_peered":"2023-10-19T10:36:55.113459+0000","last_unstale":"2023-10-19T10:37:00.323740+0000","last_undegraded":"2023-10-19T10:37:00.323740+0000","last_fullsized":"2023-10-19T10:37:00.323740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:05:40.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323207+0000","last_change":"2023-10-19T10:36:54.309423+0000","last_active":"2023-10-19T10:37:00.323207+0000","last_peered":"2023-10-19T10:37:00.323207+0000","last_clean":"2023-10-19T10:37:00.323207+0000","last_became_active":"2023-10-19T10:36:54.309122+0000","last_became_peered":"2023-10-19T10:36:54.309122+0000","last_unstale":"2023-10-19T10:37:00.323207+0000","last_undegraded":"2023-10-19T10:37:00.323207+0000","last_fullsized":"2023-10-19T10:37:00.323207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:47.999447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324166+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:37:00.324166+0000","last_peered":"2023-10-19T10:37:00.324166+0000","last_clean":"2023-10-19T10:37:00.324166+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:37:00.324166+0000","last_undegraded":"2023-10-19T10:37:00.324166+0000","last_fullsized":"2023-10-19T10:37:00.324166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098587+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:37:00.098587+0000","last_peered":"2023-10-19T10:37:00.098587+0000","last_clean":"2023-10-19T10:37:00.098587+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:37:00.098587+0000","last_undegraded":"2023-10-19T10:37:00.098587+0000","last_fullsized":"2023-10-19T10:37:00.098587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:14.745066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098140+0000","last_change":"2023-10-19T10:36:59.359383+0000","last_active":"2023-10-19T10:37:00.098140+0000","last_peered":"2023-10-19T10:37:00.098140+0000","last_clean":"2023-10-19T10:37:00.098140+0000","last_became_active":"2023-10-19T10:36:59.358978+0000","last_became_peered":"2023-10-19T10:36:59.358978+0000","last_unstale":"2023-10-19T10:37:00.098140+0000","last_undegraded":"2023-10-19T10:37:00.098140+0000","last_fullsized":"2023-10-19T10:37:00.098140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097806+0000","last_change":"2023-10-19T10:36:55.111833+0000","last_active":"2023-10-19T10:37:00.097806+0000","last_peered":"2023-10-19T10:37:00.097806+0000","last_clean":"2023-10-19T10:37:00.097806+0000","last_became_active":"2023-10-19T10:36:55.111558+0000","last_became_peered":"2023-10-19T10:36:55.111558+0000","last_unstale":"2023-10-19T10:37:00.097806+0000","last_undegraded":"2023-10-19T10:37:00.097806+0000","last_fullsized":"2023-10-19T10:37:00.097806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323791+0000","last_change":"2023-10-19T10:36:54.304575+0000","last_active":"2023-10-19T10:37:00.323791+0000","last_peered":"2023-10-19T10:37:00.323791+0000","last_clean":"2023-10-19T10:37:00.323791+0000","last_became_active":"2023-10-19T10:36:54.304001+0000","last_became_peered":"2023-10-19T10:36:54.304001+0000","last_unstale":"2023-10-19T10:37:00.323791+0000","last_undegraded":"2023-10-19T10:37:00.323791+0000","last_fullsized":"2023-10-19T10:37:00.323791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099327+0000","last_change":"2023-10-19T10:36:59.347266+0000","last_active":"2023-10-19T10:37:00.099327+0000","last_peered":"2023-10-19T10:37:00.099327+0000","last_clean":"2023-10-19T10:37:00.099327+0000","last_became_active":"2023-10-19T10:36:59.346858+0000","last_became_peered":"2023-10-19T10:36:59.346858+0000","last_unstale":"2023-10-19T10:37:00.099327+0000","last_undegraded":"2023-10-19T10:37:00.099327+0000","last_fullsized":"2023-10-19T10:37:00.099327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:30.546568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099297+0000","last_change":"2023-10-19T10:36:55.107822+0000","last_active":"2023-10-19T10:37:00.099297+0000","last_peered":"2023-10-19T10:37:00.099297+0000","last_clean":"2023-10-19T10:37:00.099297+0000","last_became_active":"2023-10-19T10:36:55.107534+0000","last_became_peered":"2023-10-19T10:36:55.107534+0000","last_unstale":"2023-10-19T10:37:00.099297+0000","last_undegraded":"2023-10-19T10:37:00.099297+0000","last_fullsized":"2023-10-19T10:37:00.099297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:51:56.439816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098149+0000","last_change":"2023-10-19T10:36:54.306612+0000","last_active":"2023-10-19T10:37:00.098149+0000","last_peered":"2023-10-19T10:37:00.098149+0000","last_clean":"2023-10-19T10:37:00.098149+0000","last_became_active":"2023-10-19T10:36:54.306357+0000","last_became_peered":"2023-10-19T10:36:54.306357+0000","last_unstale":"2023-10-19T10:37:00.098149+0000","last_undegraded":"2023-10-19T10:37:00.098149+0000","last_fullsized":"2023-10-19T10:37:00.098149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:37:13.605874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324569+0000","last_change":"2023-10-19T10:36:59.352368+0000","last_active":"2023-10-19T10:37:00.324569+0000","last_peered":"2023-10-19T10:37:00.324569+0000","last_clean":"2023-10-19T10:37:00.324569+0000","last_became_active":"2023-10-19T10:36:59.352002+0000","last_became_peered":"2023-10-19T10:36:59.352002+0000","last_unstale":"2023-10-19T10:37:00.324569+0000","last_undegraded":"2023-10-19T10:37:00.324569+0000","last_fullsized":"2023-10-19T10:37:00.324569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:05.263646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099519+0000","last_change":"2023-10-19T10:36:55.110179+0000","last_active":"2023-10-19T10:37:00.099519+0000","last_peered":"2023-10-19T10:37:00.099519+0000","last_clean":"2023-10-19T10:37:00.099519+0000","last_became_active":"2023-10-19T10:36:55.109756+0000","last_became_peered":"2023-10-19T10:36:55.109756+0000","last_unstale":"2023-10-19T10:37:00.099519+0000","last_undegraded":"2023-10-19T10:37:00.099519+0000","last_fullsized":"2023-10-19T10:37:00.099519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:55:21.657235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323576+0000","last_change":"2023-10-19T10:36:54.306427+0000","last_active":"2023-10-19T10:37:00.323576+0000","last_peered":"2023-10-19T10:37:00.323576+0000","last_clean":"2023-10-19T10:37:00.323576+0000","last_became_active":"2023-10-19T10:36:54.306033+0000","last_became_peered":"2023-10-19T10:36:54.306033+0000","last_unstale":"2023-10-19T10:37:00.323576+0000","last_undegraded":"2023-10-19T10:37:00.323576+0000","last_fullsized":"2023-10-19T10:37:00.323576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:08:11.353560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097356+0000","last_change":"2023-10-19T10:36:59.353562+0000","last_active":"2023-10-19T10:37:00.097356+0000","last_peered":"2023-10-19T10:37:00.097356+0000","last_clean":"2023-10-19T10:37:00.097356+0000","last_became_active":"2023-10-19T10:36:59.353204+0000","last_became_peered":"2023-10-19T10:36:59.353204+0000","last_unstale":"2023-10-19T10:37:00.097356+0000","last_undegraded":"2023-10-19T10:37:00.097356+0000","last_fullsized":"2023-10-19T10:37:00.097356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099281+0000","last_change":"2023-10-19T10:36:55.115534+0000","last_active":"2023-10-19T10:37:00.099281+0000","last_peered":"2023-10-19T10:37:00.099281+0000","last_clean":"2023-10-19T10:37:00.099281+0000","last_became_active":"2023-10-19T10:36:55.114049+0000","last_became_peered":"2023-10-19T10:36:55.114049+0000","last_unstale":"2023-10-19T10:37:00.099281+0000","last_undegraded":"2023-10-19T10:37:00.099281+0000","last_fullsized":"2023-10-19T10:37:00.099281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:22:31.265043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323955+0000","last_change":"2023-10-19T10:36:54.310069+0000","last_active":"2023-10-19T10:37:00.323955+0000","last_peered":"2023-10-19T10:37:00.323955+0000","last_clean":"2023-10-19T10:37:00.323955+0000","last_became_active":"2023-10-19T10:36:54.309784+0000","last_became_peered":"2023-10-19T10:36:54.309784+0000","last_unstale":"2023-10-19T10:37:00.323955+0000","last_undegraded":"2023-10-19T10:37:00.323955+0000","last_fullsized":"2023-10-19T10:37:00.323955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323321+0000","last_change":"2023-10-19T10:36:59.351003+0000","last_active":"2023-10-19T10:37:00.323321+0000","last_peered":"2023-10-19T10:37:00.323321+0000","last_clean":"2023-10-19T10:37:00.323321+0000","last_became_active":"2023-10-19T10:36:59.349597+0000","last_became_peered":"2023-10-19T10:36:59.349597+0000","last_unstale":"2023-10-19T10:37:00.323321+0000","last_undegraded":"2023-10-19T10:37:00.323321+0000","last_fullsized":"2023-10-19T10:37:00.323321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:53:25.552100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277872+0000","last_change":"2023-10-19T10:36:55.118959+0000","last_active":"2023-10-19T10:37:00.277872+0000","last_peered":"2023-10-19T10:37:00.277872+0000","last_clean":"2023-10-19T10:37:00.277872+0000","last_became_active":"2023-10-19T10:36:55.116867+0000","last_became_peered":"2023-10-19T10:36:55.116867+0000","last_unstale":"2023-10-19T10:37:00.277872+0000","last_undegraded":"2023-10-19T10:37:00.277872+0000","last_fullsized":"2023-10-19T10:37:00.277872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324124+0000","last_change":"2023-10-19T10:36:54.305145+0000","last_active":"2023-10-19T10:37:00.324124+0000","last_peered":"2023-10-19T10:37:00.324124+0000","last_clean":"2023-10-19T10:37:00.324124+0000","last_became_active":"2023-10-19T10:36:54.304599+0000","last_became_peered":"2023-10-19T10:36:54.304599+0000","last_unstale":"2023-10-19T10:37:00.324124+0000","last_undegraded":"2023-10-19T10:37:00.324124+0000","last_fullsized":"2023-10-19T10:37:00.324124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323492+0000","last_change":"2023-10-19T10:36:59.348091+0000","last_active":"2023-10-19T10:37:00.323492+0000","last_peered":"2023-10-19T10:37:00.323492+0000","last_clean":"2023-10-19T10:37:00.323492+0000","last_became_active":"2023-10-19T10:36:59.346180+0000","last_became_peered":"2023-10-19T10:36:59.346180+0000","last_unstale":"2023-10-19T10:37:00.323492+0000","last_undegraded":"2023-10-19T10:37:00.323492+0000","last_fullsized":"2023-10-19T10:37:00.323492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:42:00.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323299+0000","last_change":"2023-10-19T10:36:55.114648+0000","last_active":"2023-10-19T10:37:00.323299+0000","last_peered":"2023-10-19T10:37:00.323299+0000","last_clean":"2023-10-19T10:37:00.323299+0000","last_became_active":"2023-10-19T10:36:55.114149+0000","last_became_peered":"2023-10-19T10:36:55.114149+0000","last_unstale":"2023-10-19T10:37:00.323299+0000","last_undegraded":"2023-10-19T10:37:00.323299+0000","last_fullsized":"2023-10-19T10:37:00.323299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:25:33.396338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098964+0000","last_change":"2023-10-19T10:36:54.309069+0000","last_active":"2023-10-19T10:37:00.098964+0000","last_peered":"2023-10-19T10:37:00.098964+0000","last_clean":"2023-10-19T10:37:00.098964+0000","last_became_active":"2023-10-19T10:36:54.308767+0000","last_became_peered":"2023-10-19T10:36:54.308767+0000","last_unstale":"2023-10-19T10:37:00.098964+0000","last_undegraded":"2023-10-19T10:37:00.098964+0000","last_fullsized":"2023-10-19T10:37:00.098964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323054+0000","last_change":"2023-10-19T10:36:59.349735+0000","last_active":"2023-10-19T10:37:00.323054+0000","last_peered":"2023-10-19T10:37:00.323054+0000","last_clean":"2023-10-19T10:37:00.323054+0000","last_became_active":"2023-10-19T10:36:59.349327+0000","last_became_peered":"2023-10-19T10:36:59.349327+0000","last_unstale":"2023-10-19T10:37:00.323054+0000","last_undegraded":"2023-10-19T10:37:00.323054+0000","last_fullsized":"2023-10-19T10:37:00.323054+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:51:57.082676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097759+0000","last_change":"2023-10-19T10:36:55.111295+0000","last_active":"2023-10-19T10:37:00.097759+0000","last_peered":"2023-10-19T10:37:00.097759+0000","last_clean":"2023-10-19T10:37:00.097759+0000","last_became_active":"2023-10-19T10:36:55.111016+0000","last_became_peered":"2023-10-19T10:36:55.111016+0000","last_unstale":"2023-10-19T10:37:00.097759+0000","last_undegraded":"2023-10-19T10:37:00.097759+0000","last_fullsized":"2023-10-19T10:37:00.097759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:01:05.121484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098592+0000","last_change":"2023-10-19T10:36:54.308800+0000","last_active":"2023-10-19T10:37:00.098592+0000","last_peered":"2023-10-19T10:37:00.098592+0000","last_clean":"2023-10-19T10:37:00.098592+0000","last_became_active":"2023-10-19T10:36:54.307919+0000","last_became_peered":"2023-10-19T10:36:54.307919+0000","last_unstale":"2023-10-19T10:37:00.098592+0000","last_undegraded":"2023-10-19T10:37:00.098592+0000","last_fullsized":"2023-10-19T10:37:00.098592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:00.862675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322763+0000","last_change":"2023-10-19T10:36:59.350698+0000","last_active":"2023-10-19T10:37:00.322763+0000","last_peered":"2023-10-19T10:37:00.322763+0000","last_clean":"2023-10-19T10:37:00.322763+0000","last_became_active":"2023-10-19T10:36:59.350331+0000","last_became_peered":"2023-10-19T10:36:59.350331+0000","last_unstale":"2023-10-19T10:37:00.322763+0000","last_undegraded":"2023-10-19T10:37:00.322763+0000","last_fullsized":"2023-10-19T10:37:00.322763+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:44.911501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323455+0000","last_change":"2023-10-19T10:36:55.110357+0000","last_active":"2023-10-19T10:37:00.323455+0000","last_peered":"2023-10-19T10:37:00.323455+0000","last_clean":"2023-10-19T10:37:00.323455+0000","last_became_active":"2023-10-19T10:36:55.110058+0000","last_became_peered":"2023-10-19T10:36:55.110058+0000","last_unstale":"2023-10-19T10:37:00.323455+0000","last_undegraded":"2023-10-19T10:37:00.323455+0000","last_fullsized":"2023-10-19T10:37:00.323455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:39.760467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098977+0000","last_change":"2023-10-19T10:36:54.305854+0000","last_active":"2023-10-19T10:37:00.098977+0000","last_peered":"2023-10-19T10:37:00.098977+0000","last_clean":"2023-10-19T10:37:00.098977+0000","last_became_active":"2023-10-19T10:36:54.305559+0000","last_became_peered":"2023-10-19T10:36:54.305559+0000","last_unstale":"2023-10-19T10:37:00.098977+0000","last_undegraded":"2023-10-19T10:37:00.098977+0000","last_fullsized":"2023-10-19T10:37:00.098977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097838+0000","last_change":"2023-10-19T10:36:59.359644+0000","last_active":"2023-10-19T10:37:00.097838+0000","last_peered":"2023-10-19T10:37:00.097838+0000","last_clean":"2023-10-19T10:37:00.097838+0000","last_became_active":"2023-10-19T10:36:59.359083+0000","last_became_peered":"2023-10-19T10:36:59.359083+0000","last_unstale":"2023-10-19T10:37:00.097838+0000","last_undegraded":"2023-10-19T10:37:00.097838+0000","last_fullsized":"2023-10-19T10:37:00.097838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324477+0000","last_change":"2023-10-19T10:36:59.349191+0000","last_active":"2023-10-19T10:37:00.324477+0000","last_peered":"2023-10-19T10:37:00.324477+0000","last_clean":"2023-10-19T10:37:00.324477+0000","last_became_active":"2023-10-19T10:36:59.348388+0000","last_became_peered":"2023-10-19T10:36:59.348388+0000","last_unstale":"2023-10-19T10:37:00.324477+0000","last_undegraded":"2023-10-19T10:37:00.324477+0000","last_fullsized":"2023-10-19T10:37:00.324477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323413+0000","last_change":"2023-10-19T10:36:54.311108+0000","last_active":"2023-10-19T10:37:00.323413+0000","last_peered":"2023-10-19T10:37:00.323413+0000","last_clean":"2023-10-19T10:37:00.323413+0000","last_became_active":"2023-10-19T10:36:54.310804+0000","last_became_peered":"2023-10-19T10:36:54.310804+0000","last_unstale":"2023-10-19T10:37:00.323413+0000","last_undegraded":"2023-10-19T10:37:00.323413+0000","last_fullsized":"2023-10-19T10:37:00.323413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:42:24.153777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324424+0000","last_change":"2023-10-19T10:36:55.108985+0000","last_active":"2023-10-19T10:37:00.324424+0000","last_peered":"2023-10-19T10:37:00.324424+0000","last_clean":"2023-10-19T10:37:00.324424+0000","last_became_active":"2023-10-19T10:36:55.108726+0000","last_became_peered":"2023-10-19T10:36:55.108726+0000","last_unstale":"2023-10-19T10:37:00.324424+0000","last_undegraded":"2023-10-19T10:37:00.324424+0000","last_fullsized":"2023-10-19T10:37:00.324424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:29.622437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324136+0000","last_change":"2023-10-19T10:36:59.352844+0000","last_active":"2023-10-19T10:37:00.324136+0000","last_peered":"2023-10-19T10:37:00.324136+0000","last_clean":"2023-10-19T10:37:00.324136+0000","last_became_active":"2023-10-19T10:36:59.352451+0000","last_became_peered":"2023-10-19T10:36:59.352451+0000","last_unstale":"2023-10-19T10:37:00.324136+0000","last_undegraded":"2023-10-19T10:37:00.324136+0000","last_fullsized":"2023-10-19T10:37:00.324136+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:21.477523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098487+0000","last_change":"2023-10-19T10:36:54.310021+0000","last_active":"2023-10-19T10:37:00.098487+0000","last_peered":"2023-10-19T10:37:00.098487+0000","last_clean":"2023-10-19T10:37:00.098487+0000","last_became_active":"2023-10-19T10:36:54.309686+0000","last_became_peered":"2023-10-19T10:36:54.309686+0000","last_unstale":"2023-10-19T10:37:00.098487+0000","last_undegraded":"2023-10-19T10:37:00.098487+0000","last_fullsized":"2023-10-19T10:37:00.098487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:25.891199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278441+0000","last_change":"2023-10-19T10:36:55.118465+0000","last_active":"2023-10-19T10:37:00.278441+0000","last_peered":"2023-10-19T10:37:00.278441+0000","last_clean":"2023-10-19T10:37:00.278441+0000","last_became_active":"2023-10-19T10:36:55.116202+0000","last_became_peered":"2023-10-19T10:36:55.116202+0000","last_unstale":"2023-10-19T10:37:00.278441+0000","last_undegraded":"2023-10-19T10:37:00.278441+0000","last_fullsized":"2023-10-19T10:37:00.278441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277600+0000","last_change":"2023-10-19T10:36:54.306522+0000","last_active":"2023-10-19T10:37:00.277600+0000","last_peered":"2023-10-19T10:37:00.277600+0000","last_clean":"2023-10-19T10:37:00.277600+0000","last_became_active":"2023-10-19T10:36:54.306147+0000","last_became_peered":"2023-10-19T10:36:54.306147+0000","last_unstale":"2023-10-19T10:37:00.277600+0000","last_undegraded":"2023-10-19T10:37:00.277600+0000","last_fullsized":"2023-10-19T10:37:00.277600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:38.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324371+0000","last_change":"2023-10-19T10:36:55.110979+0000","last_active":"2023-10-19T10:37:00.324371+0000","last_peered":"2023-10-19T10:37:00.324371+0000","last_clean":"2023-10-19T10:37:00.324371+0000","last_became_active":"2023-10-19T10:36:55.109937+0000","last_became_peered":"2023-10-19T10:36:55.109937+0000","last_unstale":"2023-10-19T10:37:00.324371+0000","last_undegraded":"2023-10-19T10:37:00.324371+0000","last_fullsized":"2023-10-19T10:37:00.324371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323485+0000","last_change":"2023-10-19T10:36:59.355501+0000","last_active":"2023-10-19T10:37:00.323485+0000","last_peered":"2023-10-19T10:37:00.323485+0000","last_clean":"2023-10-19T10:37:00.323485+0000","last_became_active":"2023-10-19T10:36:59.353627+0000","last_became_peered":"2023-10-19T10:36:59.353627+0000","last_unstale":"2023-10-19T10:37:00.323485+0000","last_undegraded":"2023-10-19T10:37:00.323485+0000","last_fullsized":"2023-10-19T10:37:00.323485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:30.933353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098816+0000","last_change":"2023-10-19T10:36:54.308041+0000","last_active":"2023-10-19T10:37:00.098816+0000","last_peered":"2023-10-19T10:37:00.098816+0000","last_clean":"2023-10-19T10:37:00.098816+0000","last_became_active":"2023-10-19T10:36:54.307752+0000","last_became_peered":"2023-10-19T10:36:54.307752+0000","last_unstale":"2023-10-19T10:37:00.098816+0000","last_undegraded":"2023-10-19T10:37:00.098816+0000","last_fullsized":"2023-10-19T10:37:00.098816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323551+0000","last_change":"2023-10-19T10:36:55.115058+0000","last_active":"2023-10-19T10:37:00.323551+0000","last_peered":"2023-10-19T10:37:00.323551+0000","last_clean":"2023-10-19T10:37:00.323551+0000","last_became_active":"2023-10-19T10:36:55.114729+0000","last_became_peered":"2023-10-19T10:36:55.114729+0000","last_unstale":"2023-10-19T10:37:00.323551+0000","last_undegraded":"2023-10-19T10:37:00.323551+0000","last_fullsized":"2023-10-19T10:37:00.323551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324110+0000","last_change":"2023-10-19T10:36:59.356762+0000","last_active":"2023-10-19T10:37:00.324110+0000","last_peered":"2023-10-19T10:37:00.324110+0000","last_clean":"2023-10-19T10:37:00.324110+0000","last_became_active":"2023-10-19T10:36:59.356422+0000","last_became_peered":"2023-10-19T10:36:59.356422+0000","last_unstale":"2023-10-19T10:37:00.324110+0000","last_undegraded":"2023-10-19T10:37:00.324110+0000","last_fullsized":"2023-10-19T10:37:00.324110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323028+0000","last_change":"2023-10-19T10:36:55.108618+0000","last_active":"2023-10-19T10:37:00.323028+0000","last_peered":"2023-10-19T10:37:00.323028+0000","last_clean":"2023-10-19T10:37:00.323028+0000","last_became_active":"2023-10-19T10:36:55.108364+0000","last_became_peered":"2023-10-19T10:36:55.108364+0000","last_unstale":"2023-10-19T10:37:00.323028+0000","last_undegraded":"2023-10-19T10:37:00.323028+0000","last_fullsized":"2023-10-19T10:37:00.323028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323784+0000","last_change":"2023-10-19T10:36:55.114936+0000","last_active":"2023-10-19T10:37:00.323784+0000","last_peered":"2023-10-19T10:37:00.323784+0000","last_clean":"2023-10-19T10:37:00.323784+0000","last_became_active":"2023-10-19T10:36:55.114673+0000","last_became_peered":"2023-10-19T10:36:55.114673+0000","last_unstale":"2023-10-19T10:37:00.323784+0000","last_undegraded":"2023-10-19T10:37:00.323784+0000","last_fullsized":"2023-10-19T10:37:00.323784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098152+0000","last_change":"2023-10-19T10:36:55.107446+0000","last_active":"2023-10-19T10:37:00.098152+0000","last_peered":"2023-10-19T10:37:00.098152+0000","last_clean":"2023-10-19T10:37:00.098152+0000","last_became_active":"2023-10-19T10:36:55.107185+0000","last_became_peered":"2023-10-19T10:36:55.107185+0000","last_unstale":"2023-10-19T10:37:00.098152+0000","last_undegraded":"2023-10-19T10:37:00.098152+0000","last_fullsized":"2023-10-19T10:37:00.098152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:36:13.224759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323701+0000","last_change":"2023-10-19T10:36:55.108586+0000","last_active":"2023-10-19T10:37:00.323701+0000","last_peered":"2023-10-19T10:37:00.323701+0000","last_clean":"2023-10-19T10:37:00.323701+0000","last_became_active":"2023-10-19T10:36:55.108308+0000","last_became_peered":"2023-10-19T10:36:55.108308+0000","last_unstale":"2023-10-19T10:37:00.323701+0000","last_undegraded":"2023-10-19T10:37:00.323701+0000","last_fullsized":"2023-10-19T10:37:00.323701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:09.378639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097485+0000","last_change":"2023-10-19T10:36:55.112395+0000","last_active":"2023-10-19T10:37:00.097485+0000","last_peered":"2023-10-19T10:37:00.097485+0000","last_clean":"2023-10-19T10:37:00.097485+0000","last_became_active":"2023-10-19T10:36:55.112101+0000","last_became_peered":"2023-10-19T10:36:55.112101+0000","last_unstale":"2023-10-19T10:37:00.097485+0000","last_undegraded":"2023-10-19T10:37:00.097485+0000","last_fullsized":"2023-10-19T10:37:00.097485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:55.114054+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:55.113749+0000","last_became_peered":"2023-10-19T10:36:55.113749+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097817+0000","last_change":"2023-10-19T10:36:55.109364+0000","last_active":"2023-10-19T10:37:00.097817+0000","last_peered":"2023-10-19T10:37:00.097817+0000","last_clean":"2023-10-19T10:37:00.097817+0000","last_became_active":"2023-10-19T10:36:55.109110+0000","last_became_peered":"2023-10-19T10:36:55.109110+0000","last_unstale":"2023-10-19T10:37:00.097817+0000","last_undegraded":"2023-10-19T10:37:00.097817+0000","last_fullsized":"2023-10-19T10:37:00.097817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097790+0000","last_change":"2023-10-19T10:36:55.111296+0000","last_active":"2023-10-19T10:37:00.097790+0000","last_peered":"2023-10-19T10:37:00.097790+0000","last_clean":"2023-10-19T10:37:00.097790+0000","last_became_active":"2023-10-19T10:36:55.111042+0000","last_became_peered":"2023-10-19T10:36:55.111042+0000","last_unstale":"2023-10-19T10:37:00.097790+0000","last_undegraded":"2023-10-19T10:37:00.097790+0000","last_fullsized":"2023-10-19T10:37:00.097790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098307+0000","last_change":"2023-10-19T10:36:55.117568+0000","last_active":"2023-10-19T10:37:00.098307+0000","last_peered":"2023-10-19T10:37:00.098307+0000","last_clean":"2023-10-19T10:37:00.098307+0000","last_became_active":"2023-10-19T10:36:55.117281+0000","last_became_peered":"2023-10-19T10:36:55.117281+0000","last_unstale":"2023-10-19T10:37:00.098307+0000","last_undegraded":"2023-10-19T10:37:00.098307+0000","last_fullsized":"2023-10-19T10:37:00.098307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323081+0000","last_change":"2023-10-19T10:36:55.114123+0000","last_active":"2023-10-19T10:37:00.323081+0000","last_peered":"2023-10-19T10:37:00.323081+0000","last_clean":"2023-10-19T10:37:00.323081+0000","last_became_active":"2023-10-19T10:36:55.113832+0000","last_became_peered":"2023-10-19T10:36:55.113832+0000","last_unstale":"2023-10-19T10:37:00.323081+0000","last_undegraded":"2023-10-19T10:37:00.323081+0000","last_fullsized":"2023-10-19T10:37:00.323081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277576+0000","last_change":"2023-10-19T10:36:55.115474+0000","last_active":"2023-10-19T10:37:00.277576+0000","last_peered":"2023-10-19T10:37:00.277576+0000","last_clean":"2023-10-19T10:37:00.277576+0000","last_became_active":"2023-10-19T10:36:55.115133+0000","last_became_peered":"2023-10-19T10:36:55.115133+0000","last_unstale":"2023-10-19T10:37:00.277576+0000","last_undegraded":"2023-10-19T10:37:00.277576+0000","last_fullsized":"2023-10-19T10:37:00.277576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098770+0000","last_change":"2023-10-19T10:36:55.117512+0000","last_active":"2023-10-19T10:37:00.098770+0000","last_peered":"2023-10-19T10:37:00.098770+0000","last_clean":"2023-10-19T10:37:00.098770+0000","last_became_active":"2023-10-19T10:36:55.117143+0000","last_became_peered":"2023-10-19T10:36:55.117143+0000","last_unstale":"2023-10-19T10:37:00.098770+0000","last_undegraded":"2023-10-19T10:37:00.098770+0000","last_fullsized":"2023-10-19T10:37:00.098770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:46.891573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099100+0000","last_change":"2023-10-19T10:36:55.106989+0000","last_active":"2023-10-19T10:37:00.099100+0000","last_peered":"2023-10-19T10:37:00.099100+0000","last_clean":"2023-10-19T10:37:00.099100+0000","last_became_active":"2023-10-19T10:36:55.106735+0000","last_became_peered":"2023-10-19T10:36:55.106735+0000","last_unstale":"2023-10-19T10:37:00.099100+0000","last_undegraded":"2023-10-19T10:37:00.099100+0000","last_fullsized":"2023-10-19T10:37:00.099100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:49:54.003217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324208+0000","last_change":"2023-10-19T10:36:55.107156+0000","last_active":"2023-10-19T10:37:00.324208+0000","last_peered":"2023-10-19T10:37:00.324208+0000","last_clean":"2023-10-19T10:37:00.324208+0000","last_became_active":"2023-10-19T10:36:55.106636+0000","last_became_peered":"2023-10-19T10:36:55.106636+0000","last_unstale":"2023-10-19T10:37:00.324208+0000","last_undegraded":"2023-10-19T10:37:00.324208+0000","last_fullsized":"2023-10-19T10:37:00.324208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:02:38.159925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098995+0000","last_change":"2023-10-19T10:36:55.102460+0000","last_active":"2023-10-19T10:37:00.098995+0000","last_peered":"2023-10-19T10:37:00.098995+0000","last_clean":"2023-10-19T10:37:00.098995+0000","last_became_active":"2023-10-19T10:36:55.102197+0000","last_became_peered":"2023-10-19T10:36:55.102197+0000","last_unstale":"2023-10-19T10:37:00.098995+0000","last_undegraded":"2023-10-19T10:37:00.098995+0000","last_fullsized":"2023-10-19T10:37:00.098995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:18:06.083802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278315+0000","last_change":"2023-10-19T10:36:55.118142+0000","last_active":"2023-10-19T10:37:00.278315+0000","last_peered":"2023-10-19T10:37:00.278315+0000","last_clean":"2023-10-19T10:37:00.278315+0000","last_became_active":"2023-10-19T10:36:55.116529+0000","last_became_peered":"2023-10-19T10:36:55.116529+0000","last_unstale":"2023-10-19T10:37:00.278315+0000","last_undegraded":"2023-10-19T10:37:00.278315+0000","last_fullsized":"2023-10-19T10:37:00.278315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097588+0000","last_change":"2023-10-19T10:36:55.110137+0000","last_active":"2023-10-19T10:37:00.097588+0000","last_peered":"2023-10-19T10:37:00.097588+0000","last_clean":"2023-10-19T10:37:00.097588+0000","last_became_active":"2023-10-19T10:36:55.109003+0000","last_became_peered":"2023-10-19T10:36:55.109003+0000","last_unstale":"2023-10-19T10:37:00.097588+0000","last_undegraded":"2023-10-19T10:37:00.097588+0000","last_fullsized":"2023-10-19T10:37:00.097588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:38.218180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098351+0000","last_change":"2023-10-19T10:36:55.115362+0000","last_active":"2023-10-19T10:37:00.098351+0000","last_peered":"2023-10-19T10:37:00.098351+0000","last_clean":"2023-10-19T10:37:00.098351+0000","last_became_active":"2023-10-19T10:36:55.114806+0000","last_became_peered":"2023-10-19T10:36:55.114806+0000","last_unstale":"2023-10-19T10:37:00.098351+0000","last_undegraded":"2023-10-19T10:37:00.098351+0000","last_fullsized":"2023-10-19T10:37:00.098351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323850+0000","last_change":"2023-10-19T10:36:55.106573+0000","last_active":"2023-10-19T10:37:00.323850+0000","last_peered":"2023-10-19T10:37:00.323850+0000","last_clean":"2023-10-19T10:37:00.323850+0000","last_became_active":"2023-10-19T10:36:55.106293+0000","last_became_peered":"2023-10-19T10:36:55.106293+0000","last_unstale":"2023-10-19T10:37:00.323850+0000","last_undegraded":"2023-10-19T10:37:00.323850+0000","last_fullsized":"2023-10-19T10:37:00.323850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323255+0000","last_change":"2023-10-19T10:36:55.116400+0000","last_active":"2023-10-19T10:37:00.323255+0000","last_peered":"2023-10-19T10:37:00.323255+0000","last_clean":"2023-10-19T10:37:00.323255+0000","last_became_active":"2023-10-19T10:36:55.116142+0000","last_became_peered":"2023-10-19T10:36:55.116142+0000","last_unstale":"2023-10-19T10:37:00.323255+0000","last_undegraded":"2023-10-19T10:37:00.323255+0000","last_fullsized":"2023-10-19T10:37:00.323255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:00:11.139684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323751+0000","last_change":"2023-10-19T10:36:55.102152+0000","last_active":"2023-10-19T10:37:00.323751+0000","last_peered":"2023-10-19T10:37:00.323751+0000","last_clean":"2023-10-19T10:37:00.323751+0000","last_became_active":"2023-10-19T10:36:55.101878+0000","last_became_peered":"2023-10-19T10:36:55.101878+0000","last_unstale":"2023-10-19T10:37:00.323751+0000","last_undegraded":"2023-10-19T10:37:00.323751+0000","last_fullsized":"2023-10-19T10:37:00.323751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:27:50.253900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099451+0000","last_change":"2023-10-19T10:36:55.111165+0000","last_active":"2023-10-19T10:37:00.099451+0000","last_peered":"2023-10-19T10:37:00.099451+0000","last_clean":"2023-10-19T10:37:00.099451+0000","last_became_active":"2023-10-19T10:36:55.110828+0000","last_became_peered":"2023-10-19T10:36:55.110828+0000","last_unstale":"2023-10-19T10:37:00.099451+0000","last_undegraded":"2023-10-19T10:37:00.099451+0000","last_fullsized":"2023-10-19T10:37:00.099451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:19:39.989618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324638+0000","last_change":"2023-10-19T10:36:55.110928+0000","last_active":"2023-10-19T10:37:00.324638+0000","last_peered":"2023-10-19T10:37:00.324638+0000","last_clean":"2023-10-19T10:37:00.324638+0000","last_became_active":"2023-10-19T10:36:55.110647+0000","last_became_peered":"2023-10-19T10:36:55.110647+0000","last_unstale":"2023-10-19T10:37:00.324638+0000","last_undegraded":"2023-10-19T10:37:00.324638+0000","last_fullsized":"2023-10-19T10:37:00.324638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:17.439266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278188+0000","last_change":"2023-10-19T10:36:55.118334+0000","last_active":"2023-10-19T10:37:00.278188+0000","last_peered":"2023-10-19T10:37:00.278188+0000","last_clean":"2023-10-19T10:37:00.278188+0000","last_became_active":"2023-10-19T10:36:55.116957+0000","last_became_peered":"2023-10-19T10:36:55.116957+0000","last_unstale":"2023-10-19T10:37:00.278188+0000","last_undegraded":"2023-10-19T10:37:00.278188+0000","last_fullsized":"2023-10-19T10:37:00.278188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323040+0000","last_change":"2023-10-19T10:36:55.110747+0000","last_active":"2023-10-19T10:37:00.323040+0000","last_peered":"2023-10-19T10:37:00.323040+0000","last_clean":"2023-10-19T10:37:00.323040+0000","last_became_active":"2023-10-19T10:36:55.110453+0000","last_became_peered":"2023-10-19T10:36:55.110453+0000","last_unstale":"2023-10-19T10:37:00.323040+0000","last_undegraded":"2023-10-19T10:37:00.323040+0000","last_fullsized":"2023-10-19T10:37:00.323040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097980+0000","last_change":"2023-10-19T10:36:55.111146+0000","last_active":"2023-10-19T10:37:00.097980+0000","last_peered":"2023-10-19T10:37:00.097980+0000","last_clean":"2023-10-19T10:37:00.097980+0000","last_became_active":"2023-10-19T10:36:55.110894+0000","last_became_peered":"2023-10-19T10:36:55.110894+0000","last_unstale":"2023-10-19T10:37:00.097980+0000","last_undegraded":"2023-10-19T10:37:00.097980+0000","last_fullsized":"2023-10-19T10:37:00.097980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097815+0000","last_change":"2023-10-19T10:36:55.109335+0000","last_active":"2023-10-19T10:37:00.097815+0000","last_peered":"2023-10-19T10:37:00.097815+0000","last_clean":"2023-10-19T10:37:00.097815+0000","last_became_active":"2023-10-19T10:36:55.109002+0000","last_became_peered":"2023-10-19T10:36:55.109002+0000","last_unstale":"2023-10-19T10:37:00.097815+0000","last_undegraded":"2023-10-19T10:37:00.097815+0000","last_fullsized":"2023-10-19T10:37:00.097815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:32:10.062310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278344+0000","last_change":"2023-10-19T10:36:55.119014+0000","last_active":"2023-10-19T10:37:00.278344+0000","last_peered":"2023-10-19T10:37:00.278344+0000","last_clean":"2023-10-19T10:37:00.278344+0000","last_became_active":"2023-10-19T10:36:55.118470+0000","last_became_peered":"2023-10-19T10:36:55.118470+0000","last_unstale":"2023-10-19T10:37:00.278344+0000","last_undegraded":"2023-10-19T10:37:00.278344+0000","last_fullsized":"2023-10-19T10:37:00.278344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323295+0000","last_change":"2023-10-19T10:36:55.114470+0000","last_active":"2023-10-19T10:37:00.323295+0000","last_peered":"2023-10-19T10:37:00.323295+0000","last_clean":"2023-10-19T10:37:00.323295+0000","last_became_active":"2023-10-19T10:36:55.113809+0000","last_became_peered":"2023-10-19T10:36:55.113809+0000","last_unstale":"2023-10-19T10:37:00.323295+0000","last_undegraded":"2023-10-19T10:37:00.323295+0000","last_fullsized":"2023-10-19T10:37:00.323295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:22:52.211277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099367+0000","last_change":"2023-10-19T10:36:55.113901+0000","last_active":"2023-10-19T10:37:00.099367+0000","last_peered":"2023-10-19T10:37:00.099367+0000","last_clean":"2023-10-19T10:37:00.099367+0000","last_became_active":"2023-10-19T10:36:55.113629+0000","last_became_peered":"2023-10-19T10:36:55.113629+0000","last_unstale":"2023-10-19T10:37:00.099367+0000","last_undegraded":"2023-10-19T10:37:00.099367+0000","last_fullsized":"2023-10-19T10:37:00.099367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:20:21.320100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324574+0000","last_change":"2023-10-19T10:36:55.112724+0000","last_active":"2023-10-19T10:37:00.324574+0000","last_peered":"2023-10-19T10:37:00.324574+0000","last_clean":"2023-10-19T10:37:00.324574+0000","last_became_active":"2023-10-19T10:36:55.112307+0000","last_became_peered":"2023-10-19T10:36:55.112307+0000","last_unstale":"2023-10-19T10:37:00.324574+0000","last_undegraded":"2023-10-19T10:37:00.324574+0000","last_fullsized":"2023-10-19T10:37:00.324574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:51.008519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324311+0000","last_change":"2023-10-19T10:36:55.107659+0000","last_active":"2023-10-19T10:37:00.324311+0000","last_peered":"2023-10-19T10:37:00.324311+0000","last_clean":"2023-10-19T10:37:00.324311+0000","last_became_active":"2023-10-19T10:36:55.107382+0000","last_became_peered":"2023-10-19T10:36:55.107382+0000","last_unstale":"2023-10-19T10:37:00.324311+0000","last_undegraded":"2023-10-19T10:37:00.324311+0000","last_fullsized":"2023-10-19T10:37:00.324311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":294912,"data_stored":119711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":888832,"data_stored":710571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:37:19.137 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:37:19.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.274+0000 7fb612e41640 1 Processor -- start 2023-10-19T10:37:19.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.274+0000 7fb612e41640 1 -- start start 2023-10-19T10:37:19.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb612e41640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb60c159280 0x7fb60c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb612e41640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb612e41640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c15d2d0 con 0x7fb60c0cd7d0 2023-10-19T10:37:19.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb612e41640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c15a820 con 0x7fb60c159280 2023-10-19T10:37:19.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb612e41640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0ca480 con 0x7fb60c159e90 2023-10-19T10:37:19.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb60bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb60c0cd7d0 0x7fb60c0c8660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33526/0 (socket says 172.21.15.50:33526) 2023-10-19T10:37:19.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb60bfff640 1 -- 172.21.15.50:0/3303672377 learned_addr learned my addr 172.21.15.50:0/3303672377 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:19.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb610bdd640 1 --2- 172.21.15.50:0/3303672377 >> v2:172.21.15.183:3300/0 conn(0x7fb60c159280 0x7fb60c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb6113de640 1 --2- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb60b7fe640 1 -- 172.21.15.50:0/3303672377 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886014598 0 0) 0x7fb60c15d2d0 con 0x7fb60c0cd7d0 2023-10-19T10:37:19.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb60b7fe640 1 -- 172.21.15.50:0/3303672377 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5f8003520 con 0x7fb60c0cd7d0 2023-10-19T10:37:19.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb610bdd640 1 -- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 msgr2=0x7fb60c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:19.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb610bdd640 1 --2- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb610bdd640 1 -- 172.21.15.50:0/3303672377 >> v1:172.21.15.50:6789/0 conn(0x7fb60c0cd7d0 legacy=0x7fb60c0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.275+0000 7fb610bdd640 1 -- 172.21.15.50:0/3303672377 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c159990 con 0x7fb60c159280 2023-10-19T10:37:19.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.276+0000 7fb610bdd640 1 --2- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c159280 0x7fb60c15cd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6040097a0 tx=0x7fb60402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bf7a7b9b095061 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.276+0000 7fb60b7fe640 1 -- 172.21.15.50:0/3303672377 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604031830 con 0x7fb60c159280 2023-10-19T10:37:19.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb60b7fe640 1 -- 172.21.15.50:0/3303672377 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604031990 con 0x7fb60c159280 2023-10-19T10:37:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb60b7fe640 1 -- 172.21.15.50:0/3303672377 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604031d00 con 0x7fb60c159280 2023-10-19T10:37:19.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 -- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c159280 msgr2=0x7fb60c15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 --2- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c159280 0x7fb60c15cd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6040097a0 tx=0x7fb60402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 -- 172.21.15.50:0/3303672377 shutdown_connections 2023-10-19T10:37:19.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 --2- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 --2- 172.21.15.50:0/3303672377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c159280 0x7fb60c15cd90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 -- 172.21.15.50:0/3303672377 >> 172.21.15.50:0/3303672377 conn(0x7fb60c069e90 msgr2=0x7fb60c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:19.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 -- 172.21.15.50:0/3303672377 shutdown_connections 2023-10-19T10:37:19.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.277+0000 7fb612e41640 1 -- 172.21.15.50:0/3303672377 wait complete. 2023-10-19T10:37:19.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.278+0000 7fb612e41640 1 Processor -- start 2023-10-19T10:37:19.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.278+0000 7fb612e41640 1 -- start start 2023-10-19T10:37:19.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb612e41640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c0cd7d0 0x7fb60c0c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb612e41640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 0x7fb60c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb612e41640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c0c2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb612e41640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c14bac0 con 0x7fb60c159280 2023-10-19T10:37:19.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb612e41640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c1f0e90 con 0x7fb60c0cd7d0 2023-10-19T10:37:19.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb612e41640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c30c0 con 0x7fb60c159e90 2023-10-19T10:37:19.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 0x7fb60c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 0x7fb60c14b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38820/0 (socket says 172.21.15.50:38820) 2023-10-19T10:37:19.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 -- 172.21.15.50:0/1252198397 learned_addr learned my addr 172.21.15.50:0/1252198397 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:19.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb610bdd640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c0cd7d0 0x7fb60c0c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb6113de640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c0c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 -- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 msgr2=0x7fb60c0c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c0c2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 -- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c0cd7d0 msgr2=0x7fb60c0c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c0cd7d0 0x7fb60c0c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb60bfff640 1 -- 172.21.15.50:0/1252198397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604009400 con 0x7fb60c159280 2023-10-19T10:37:19.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.279+0000 7fb610bdd640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c0cd7d0 0x7fb60c0c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:19.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.280+0000 7fb60bfff640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 0x7fb60c14b580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00cb70 tx=0x7fb5fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.280+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00ace0 con 0x7fb60c159280 2023-10-19T10:37:19.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.280+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00ae40 con 0x7fb60c159280 2023-10-19T10:37:19.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.280+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c0c2550 con 0x7fb60c159280 2023-10-19T10:37:19.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.280+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc012620 con 0x7fb60c159280 2023-10-19T10:37:19.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.280+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c0c2b00 con 0x7fb60c159280 2023-10-19T10:37:19.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.281+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0c19b0 con 0x7fb60c159280 2023-10-19T10:37:19.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.281+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb5fc007070 con 0x7fb60c159280 2023-10-19T10:37:19.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.282+0000 7fb6097fa640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb5ec06a050 0x7fb5ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.282+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb5fc088850 con 0x7fb60c159280 2023-10-19T10:37:19.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.282+0000 7fb610bdd640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb5ec06a050 0x7fb5ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.282+0000 7fb610bdd640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb5ec06a050 0x7fb5ec06c510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb604036000 tx=0x7fb604008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.286+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb5fc0576e0 con 0x7fb60c159280 2023-10-19T10:37:19.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.457+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb60c15d7e0 con 0x7fb5ec06a050 2023-10-19T10:37:19.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.467+0000 7fb6097fa640 1 -- 172.21.15.50:0/1252198397 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367389 (secure 0 0 0) 0x7fb604004040 con 0x7fb5ec06a050 2023-10-19T10:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:19.475 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:37:19.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb5ec06a050 msgr2=0x7fb5ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb5ec06a050 0x7fb5ec06c510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb604036000 tx=0x7fb604008030 comp rx=0 tx=0).stop 2023-10-19T10:37:19.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 msgr2=0x7fb60c14b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 0x7fb60c14b580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00cb70 tx=0x7fb5fc00a430 comp rx=0 tx=0).stop 2023-10-19T10:37:19.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 shutdown_connections 2023-10-19T10:37:19.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb5ec06a050 0x7fb5ec06c510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb60c159e90 0x7fb60c0c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb60c159280 0x7fb60c14b580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 --2- 172.21.15.50:0/1252198397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb60c0cd7d0 0x7fb60c0c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 >> 172.21.15.50:0/1252198397 conn(0x7fb60c069e90 msgr2=0x7fb60c0ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:19.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 shutdown_connections 2023-10-19T10:37:19.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.470+0000 7fb612e41640 1 -- 172.21.15.50:0/1252198397 wait complete. 2023-10-19T10:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-19T10:37:18.468770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236080,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738640,"statfs":{"total":773094113280,"available":772852367360,"internally_reserved":0,"allocated":3579904,"data_stored":2146982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003436"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096693+0000","last_change":"2023-10-19T10:36:59.359352+0000","last_active":"2023-10-19T10:37:00.096693+0000","last_peered":"2023-10-19T10:37:00.096693+0000","last_clean":"2023-10-19T10:37:00.096693+0000","last_became_active":"2023-10-19T10:36:59.358992+0000","last_became_peered":"2023-10-19T10:36:59.358992+0000","last_unstale":"2023-10-19T10:37:00.096693+0000","last_undegraded":"2023-10-19T10:37:00.096693+0000","last_fullsized":"2023-10-19T10:37:00.096693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:36:39.869930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278383+0000","last_change":"2023-10-19T10:36:54.305740+0000","last_active":"2023-10-19T10:37:00.278383+0000","last_peered":"2023-10-19T10:37:00.278383+0000","last_clean":"2023-10-19T10:37:00.278383+0000","last_became_active":"2023-10-19T10:36:54.305192+0000","last_became_peered":"2023-10-19T10:36:54.305192+0000","last_unstale":"2023-10-19T10:37:00.278383+0000","last_undegraded":"2023-10-19T10:37:00.278383+0000","last_fullsized":"2023-10-19T10:37:00.278383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097666+0000","last_change":"2023-10-19T10:36:55.108193+0000","last_active":"2023-10-19T10:37:00.097666+0000","last_peered":"2023-10-19T10:37:00.097666+0000","last_clean":"2023-10-19T10:37:00.097666+0000","last_became_active":"2023-10-19T10:36:55.107375+0000","last_became_peered":"2023-10-19T10:36:55.107375+0000","last_unstale":"2023-10-19T10:37:00.097666+0000","last_undegraded":"2023-10-19T10:37:00.097666+0000","last_fullsized":"2023-10-19T10:37:00.097666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:13:03.441540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097567+0000","last_change":"2023-10-19T10:36:59.344962+0000","last_active":"2023-10-19T10:37:00.097567+0000","last_peered":"2023-10-19T10:37:00.097567+0000","last_clean":"2023-10-19T10:37:00.097567+0000","last_became_active":"2023-10-19T10:36:59.344519+0000","last_became_peered":"2023-10-19T10:36:59.344519+0000","last_unstale":"2023-10-19T10:37:00.097567+0000","last_undegraded":"2023-10-19T10:37:00.097567+0000","last_fullsized":"2023-10-19T10:37:00.097567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:33.660694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:54.310896+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:54.310583+0000","last_became_peered":"2023-10-19T10:36:54.310583+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:56:34.283482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323668+0000","last_change":"2023-10-19T10:36:55.104115+0000","last_active":"2023-10-19T10:37:00.323668+0000","last_peered":"2023-10-19T10:37:00.323668+0000","last_clean":"2023-10-19T10:37:00.323668+0000","last_became_active":"2023-10-19T10:36:55.103785+0000","last_became_peered":"2023-10-19T10:36:55.103785+0000","last_unstale":"2023-10-19T10:37:00.323668+0000","last_undegraded":"2023-10-19T10:37:00.323668+0000","last_fullsized":"2023-10-19T10:37:00.323668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324072+0000","last_change":"2023-10-19T10:36:59.350493+0000","last_active":"2023-10-19T10:37:00.324072+0000","last_peered":"2023-10-19T10:37:00.324072+0000","last_clean":"2023-10-19T10:37:00.324072+0000","last_became_active":"2023-10-19T10:36:59.350100+0000","last_became_peered":"2023-10-19T10:36:59.350100+0000","last_unstale":"2023-10-19T10:37:00.324072+0000","last_undegraded":"2023-10-19T10:37:00.324072+0000","last_fullsized":"2023-10-19T10:37:00.324072+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324355+0000","last_change":"2023-10-19T10:36:54.307087+0000","last_active":"2023-10-19T10:37:00.324355+0000","last_peered":"2023-10-19T10:37:00.324355+0000","last_clean":"2023-10-19T10:37:00.324355+0000","last_became_active":"2023-10-19T10:36:54.306818+0000","last_became_peered":"2023-10-19T10:36:54.306818+0000","last_unstale":"2023-10-19T10:37:00.324355+0000","last_undegraded":"2023-10-19T10:37:00.324355+0000","last_fullsized":"2023-10-19T10:37:00.324355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:25.504656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098032+0000","last_change":"2023-10-19T10:36:55.111676+0000","last_active":"2023-10-19T10:37:00.098032+0000","last_peered":"2023-10-19T10:37:00.098032+0000","last_clean":"2023-10-19T10:37:00.098032+0000","last_became_active":"2023-10-19T10:36:55.110442+0000","last_became_peered":"2023-10-19T10:36:55.110442+0000","last_unstale":"2023-10-19T10:37:00.098032+0000","last_undegraded":"2023-10-19T10:37:00.098032+0000","last_fullsized":"2023-10-19T10:37:00.098032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:51:02.897898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097851+0000","last_change":"2023-10-19T10:36:59.357016+0000","last_active":"2023-10-19T10:37:00.097851+0000","last_peered":"2023-10-19T10:37:00.097851+0000","last_clean":"2023-10-19T10:37:00.097851+0000","last_became_active":"2023-10-19T10:36:59.356666+0000","last_became_peered":"2023-10-19T10:36:59.356666+0000","last_unstale":"2023-10-19T10:37:00.097851+0000","last_undegraded":"2023-10-19T10:37:00.097851+0000","last_fullsized":"2023-10-19T10:37:00.097851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324054+0000","last_change":"2023-10-19T10:36:54.306526+0000","last_active":"2023-10-19T10:37:00.324054+0000","last_peered":"2023-10-19T10:37:00.324054+0000","last_clean":"2023-10-19T10:37:00.324054+0000","last_became_active":"2023-10-19T10:36:54.306285+0000","last_became_peered":"2023-10-19T10:36:54.306285+0000","last_unstale":"2023-10-19T10:37:00.324054+0000","last_undegraded":"2023-10-19T10:37:00.324054+0000","last_fullsized":"2023-10-19T10:37:00.324054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:55.717306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098946+0000","last_change":"2023-10-19T10:36:55.115029+0000","last_active":"2023-10-19T10:37:00.098946+0000","last_peered":"2023-10-19T10:37:00.098946+0000","last_clean":"2023-10-19T10:37:00.098946+0000","last_became_active":"2023-10-19T10:36:55.114754+0000","last_became_peered":"2023-10-19T10:36:55.114754+0000","last_unstale":"2023-10-19T10:37:00.098946+0000","last_undegraded":"2023-10-19T10:37:00.098946+0000","last_fullsized":"2023-10-19T10:37:00.098946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:30:31.789571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097648+0000","last_change":"2023-10-19T10:36:59.355687+0000","last_active":"2023-10-19T10:37:00.097648+0000","last_peered":"2023-10-19T10:37:00.097648+0000","last_clean":"2023-10-19T10:37:00.097648+0000","last_became_active":"2023-10-19T10:36:59.355294+0000","last_became_peered":"2023-10-19T10:36:59.355294+0000","last_unstale":"2023-10-19T10:37:00.097648+0000","last_undegraded":"2023-10-19T10:37:00.097648+0000","last_fullsized":"2023-10-19T10:37:00.097648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:39:00.759461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278352+0000","last_change":"2023-10-19T10:36:54.307829+0000","last_active":"2023-10-19T10:37:00.278352+0000","last_peered":"2023-10-19T10:37:00.278352+0000","last_clean":"2023-10-19T10:37:00.278352+0000","last_became_active":"2023-10-19T10:36:54.307571+0000","last_became_peered":"2023-10-19T10:36:54.307571+0000","last_unstale":"2023-10-19T10:37:00.278352+0000","last_undegraded":"2023-10-19T10:37:00.278352+0000","last_fullsized":"2023-10-19T10:37:00.278352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:45.887676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323888+0000","last_change":"2023-10-19T10:36:55.112212+0000","last_active":"2023-10-19T10:37:00.323888+0000","last_peered":"2023-10-19T10:37:00.323888+0000","last_clean":"2023-10-19T10:37:00.323888+0000","last_became_active":"2023-10-19T10:36:55.111854+0000","last_became_peered":"2023-10-19T10:36:55.111854+0000","last_unstale":"2023-10-19T10:37:00.323888+0000","last_undegraded":"2023-10-19T10:37:00.323888+0000","last_fullsized":"2023-10-19T10:37:00.323888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:49:46.713742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099140+0000","last_change":"2023-10-19T10:36:59.352206+0000","last_active":"2023-10-19T10:37:00.099140+0000","last_peered":"2023-10-19T10:37:00.099140+0000","last_clean":"2023-10-19T10:37:00.099140+0000","last_became_active":"2023-10-19T10:36:59.351813+0000","last_became_peered":"2023-10-19T10:36:59.351813+0000","last_unstale":"2023-10-19T10:37:00.099140+0000","last_undegraded":"2023-10-19T10:37:00.099140+0000","last_fullsized":"2023-10-19T10:37:00.099140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:02.738858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098273+0000","last_change":"2023-10-19T10:36:54.305390+0000","last_active":"2023-10-19T10:37:00.098273+0000","last_peered":"2023-10-19T10:37:00.098273+0000","last_clean":"2023-10-19T10:37:00.098273+0000","last_became_active":"2023-10-19T10:36:54.304960+0000","last_became_peered":"2023-10-19T10:36:54.304960+0000","last_unstale":"2023-10-19T10:37:00.098273+0000","last_undegraded":"2023-10-19T10:37:00.098273+0000","last_fullsized":"2023-10-19T10:37:00.098273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:21.713184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324108+0000","last_change":"2023-10-19T10:36:55.114266+0000","last_active":"2023-10-19T10:37:00.324108+0000","last_peered":"2023-10-19T10:37:00.324108+0000","last_clean":"2023-10-19T10:37:00.324108+0000","last_became_active":"2023-10-19T10:36:55.113992+0000","last_became_peered":"2023-10-19T10:36:55.113992+0000","last_unstale":"2023-10-19T10:37:00.324108+0000","last_undegraded":"2023-10-19T10:37:00.324108+0000","last_fullsized":"2023-10-19T10:37:00.324108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:17.080140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323688+0000","last_change":"2023-10-19T10:36:59.349278+0000","last_active":"2023-10-19T10:37:00.323688+0000","last_peered":"2023-10-19T10:37:00.323688+0000","last_clean":"2023-10-19T10:37:00.323688+0000","last_became_active":"2023-10-19T10:36:59.348852+0000","last_became_peered":"2023-10-19T10:36:59.348852+0000","last_unstale":"2023-10-19T10:37:00.323688+0000","last_undegraded":"2023-10-19T10:37:00.323688+0000","last_fullsized":"2023-10-19T10:37:00.323688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:20.365692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096958+0000","last_change":"2023-10-19T10:36:54.306039+0000","last_active":"2023-10-19T10:37:00.096958+0000","last_peered":"2023-10-19T10:37:00.096958+0000","last_clean":"2023-10-19T10:37:00.096958+0000","last_became_active":"2023-10-19T10:36:54.304502+0000","last_became_peered":"2023-10-19T10:36:54.304502+0000","last_unstale":"2023-10-19T10:37:00.096958+0000","last_undegraded":"2023-10-19T10:37:00.096958+0000","last_fullsized":"2023-10-19T10:37:00.096958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:07:52.900648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278255+0000","last_change":"2023-10-19T10:36:55.114034+0000","last_active":"2023-10-19T10:37:00.278255+0000","last_peered":"2023-10-19T10:37:00.278255+0000","last_clean":"2023-10-19T10:37:00.278255+0000","last_became_active":"2023-10-19T10:36:55.113734+0000","last_became_peered":"2023-10-19T10:36:55.113734+0000","last_unstale":"2023-10-19T10:37:00.278255+0000","last_undegraded":"2023-10-19T10:37:00.278255+0000","last_fullsized":"2023-10-19T10:37:00.278255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278113+0000","last_change":"2023-10-19T10:36:59.349586+0000","last_active":"2023-10-19T10:37:00.278113+0000","last_peered":"2023-10-19T10:37:00.278113+0000","last_clean":"2023-10-19T10:37:00.278113+0000","last_became_active":"2023-10-19T10:36:59.349176+0000","last_became_peered":"2023-10-19T10:36:59.349176+0000","last_unstale":"2023-10-19T10:37:00.278113+0000","last_undegraded":"2023-10-19T10:37:00.278113+0000","last_fullsized":"2023-10-19T10:37:00.278113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323262+0000","last_change":"2023-10-19T10:36:54.310735+0000","last_active":"2023-10-19T10:37:00.323262+0000","last_peered":"2023-10-19T10:37:00.323262+0000","last_clean":"2023-10-19T10:37:00.323262+0000","last_became_active":"2023-10-19T10:36:54.309734+0000","last_became_peered":"2023-10-19T10:36:54.309734+0000","last_unstale":"2023-10-19T10:37:00.323262+0000","last_undegraded":"2023-10-19T10:37:00.323262+0000","last_fullsized":"2023-10-19T10:37:00.323262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323712+0000","last_change":"2023-10-19T10:36:55.113078+0000","last_active":"2023-10-19T10:37:00.323712+0000","last_peered":"2023-10-19T10:37:00.323712+0000","last_clean":"2023-10-19T10:37:00.323712+0000","last_became_active":"2023-10-19T10:36:55.112775+0000","last_became_peered":"2023-10-19T10:36:55.112775+0000","last_unstale":"2023-10-19T10:37:00.323712+0000","last_undegraded":"2023-10-19T10:37:00.323712+0000","last_fullsized":"2023-10-19T10:37:00.323712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:12:41.211214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322933+0000","last_change":"2023-10-19T10:36:55.105391+0000","last_active":"2023-10-19T10:37:00.322933+0000","last_peered":"2023-10-19T10:37:00.322933+0000","last_clean":"2023-10-19T10:37:00.322933+0000","last_became_active":"2023-10-19T10:36:55.105112+0000","last_became_peered":"2023-10-19T10:36:55.105112+0000","last_unstale":"2023-10-19T10:37:00.322933+0000","last_undegraded":"2023-10-19T10:37:00.322933+0000","last_fullsized":"2023-10-19T10:37:00.322933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278476+0000","last_change":"2023-10-19T10:36:54.305435+0000","last_active":"2023-10-19T10:37:00.278476+0000","last_peered":"2023-10-19T10:37:00.278476+0000","last_clean":"2023-10-19T10:37:00.278476+0000","last_became_active":"2023-10-19T10:36:54.305123+0000","last_became_peered":"2023-10-19T10:36:54.305123+0000","last_unstale":"2023-10-19T10:37:00.278476+0000","last_undegraded":"2023-10-19T10:37:00.278476+0000","last_fullsized":"2023-10-19T10:37:00.278476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323209+0000","last_change":"2023-10-19T10:36:59.350259+0000","last_active":"2023-10-19T10:37:00.323209+0000","last_peered":"2023-10-19T10:37:00.323209+0000","last_clean":"2023-10-19T10:37:00.323209+0000","last_became_active":"2023-10-19T10:36:59.349878+0000","last_became_peered":"2023-10-19T10:36:59.349878+0000","last_unstale":"2023-10-19T10:37:00.323209+0000","last_undegraded":"2023-10-19T10:37:00.323209+0000","last_fullsized":"2023-10-19T10:37:00.323209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323250+0000","last_change":"2023-10-19T10:36:55.113112+0000","last_active":"2023-10-19T10:37:00.323250+0000","last_peered":"2023-10-19T10:37:00.323250+0000","last_clean":"2023-10-19T10:37:00.323250+0000","last_became_active":"2023-10-19T10:36:55.110232+0000","last_became_peered":"2023-10-19T10:36:55.110232+0000","last_unstale":"2023-10-19T10:37:00.323250+0000","last_undegraded":"2023-10-19T10:37:00.323250+0000","last_fullsized":"2023-10-19T10:37:00.323250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324722+0000","last_change":"2023-10-19T10:36:54.310064+0000","last_active":"2023-10-19T10:37:00.324722+0000","last_peered":"2023-10-19T10:37:00.324722+0000","last_clean":"2023-10-19T10:37:00.324722+0000","last_became_active":"2023-10-19T10:36:54.309581+0000","last_became_peered":"2023-10-19T10:36:54.309581+0000","last_unstale":"2023-10-19T10:37:00.324722+0000","last_undegraded":"2023-10-19T10:37:00.324722+0000","last_fullsized":"2023-10-19T10:37:00.324722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:16.076243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098023+0000","last_change":"2023-10-19T10:36:59.355292+0000","last_active":"2023-10-19T10:37:00.098023+0000","last_peered":"2023-10-19T10:37:00.098023+0000","last_clean":"2023-10-19T10:37:00.098023+0000","last_became_active":"2023-10-19T10:36:59.354866+0000","last_became_peered":"2023-10-19T10:36:59.354866+0000","last_unstale":"2023-10-19T10:37:00.098023+0000","last_undegraded":"2023-10-19T10:37:00.098023+0000","last_fullsized":"2023-10-19T10:37:00.098023+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098814+0000","last_change":"2023-10-19T10:36:54.310722+0000","last_active":"2023-10-19T10:37:00.098814+0000","last_peered":"2023-10-19T10:37:00.098814+0000","last_clean":"2023-10-19T10:37:00.098814+0000","last_became_active":"2023-10-19T10:36:54.310410+0000","last_became_peered":"2023-10-19T10:36:54.310410+0000","last_unstale":"2023-10-19T10:37:00.098814+0000","last_undegraded":"2023-10-19T10:37:00.098814+0000","last_fullsized":"2023-10-19T10:37:00.098814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:00.221492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097163+0000","last_change":"2023-10-19T10:36:55.110762+0000","last_active":"2023-10-19T10:37:00.097163+0000","last_peered":"2023-10-19T10:37:00.097163+0000","last_clean":"2023-10-19T10:37:00.097163+0000","last_became_active":"2023-10-19T10:36:55.110503+0000","last_became_peered":"2023-10-19T10:36:55.110503+0000","last_unstale":"2023-10-19T10:37:00.097163+0000","last_undegraded":"2023-10-19T10:37:00.097163+0000","last_fullsized":"2023-10-19T10:37:00.097163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:14.103808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323380+0000","last_change":"2023-10-19T10:36:59.352779+0000","last_active":"2023-10-19T10:37:00.323380+0000","last_peered":"2023-10-19T10:37:00.323380+0000","last_clean":"2023-10-19T10:37:00.323380+0000","last_became_active":"2023-10-19T10:36:59.352362+0000","last_became_peered":"2023-10-19T10:36:59.352362+0000","last_unstale":"2023-10-19T10:37:00.323380+0000","last_undegraded":"2023-10-19T10:37:00.323380+0000","last_fullsized":"2023-10-19T10:37:00.323380+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:53.134651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:04.746420+0000","last_change":"2023-10-19T10:36:54.310368+0000","last_active":"2023-10-19T10:37:04.746420+0000","last_peered":"2023-10-19T10:37:04.746420+0000","last_clean":"2023-10-19T10:37:04.746420+0000","last_became_active":"2023-10-19T10:36:54.309745+0000","last_became_peered":"2023-10-19T10:36:54.309745+0000","last_unstale":"2023-10-19T10:37:04.746420+0000","last_undegraded":"2023-10-19T10:37:04.746420+0000","last_fullsized":"2023-10-19T10:37:04.746420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:15:27.898150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099416+0000","last_change":"2023-10-19T10:36:55.106300+0000","last_active":"2023-10-19T10:37:00.099416+0000","last_peered":"2023-10-19T10:37:00.099416+0000","last_clean":"2023-10-19T10:37:00.099416+0000","last_became_active":"2023-10-19T10:36:55.106045+0000","last_became_peered":"2023-10-19T10:36:55.106045+0000","last_unstale":"2023-10-19T10:37:00.099416+0000","last_undegraded":"2023-10-19T10:37:00.099416+0000","last_fullsized":"2023-10-19T10:37:00.099416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:28:29.050287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096752+0000","last_change":"2023-10-19T10:36:59.358801+0000","last_active":"2023-10-19T10:37:00.096752+0000","last_peered":"2023-10-19T10:37:00.096752+0000","last_clean":"2023-10-19T10:37:00.096752+0000","last_became_active":"2023-10-19T10:36:59.356906+0000","last_became_peered":"2023-10-19T10:36:59.356906+0000","last_unstale":"2023-10-19T10:37:00.096752+0000","last_undegraded":"2023-10-19T10:37:00.096752+0000","last_fullsized":"2023-10-19T10:37:00.096752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:56.099256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097964+0000","last_change":"2023-10-19T10:36:55.105397+0000","last_active":"2023-10-19T10:37:00.097964+0000","last_peered":"2023-10-19T10:37:00.097964+0000","last_clean":"2023-10-19T10:37:00.097964+0000","last_became_active":"2023-10-19T10:36:55.105016+0000","last_became_peered":"2023-10-19T10:36:55.105016+0000","last_unstale":"2023-10-19T10:37:00.097964+0000","last_undegraded":"2023-10-19T10:37:00.097964+0000","last_fullsized":"2023-10-19T10:37:00.097964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:13.604819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323795+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:37:00.323795+0000","last_peered":"2023-10-19T10:37:00.323795+0000","last_clean":"2023-10-19T10:37:00.323795+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:37:00.323795+0000","last_undegraded":"2023-10-19T10:37:00.323795+0000","last_fullsized":"2023-10-19T10:37:00.323795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:47:38.405947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278495+0000","last_change":"2023-10-19T10:36:54.301501+0000","last_active":"2023-10-19T10:37:00.278495+0000","last_peered":"2023-10-19T10:37:00.278495+0000","last_clean":"2023-10-19T10:37:00.278495+0000","last_became_active":"2023-10-19T10:36:54.301193+0000","last_became_peered":"2023-10-19T10:36:54.301193+0000","last_unstale":"2023-10-19T10:37:00.278495+0000","last_undegraded":"2023-10-19T10:37:00.278495+0000","last_fullsized":"2023-10-19T10:37:00.278495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323709+0000","last_change":"2023-10-19T10:36:59.351528+0000","last_active":"2023-10-19T10:37:00.323709+0000","last_peered":"2023-10-19T10:37:00.323709+0000","last_clean":"2023-10-19T10:37:00.323709+0000","last_became_active":"2023-10-19T10:36:59.351102+0000","last_became_peered":"2023-10-19T10:36:59.351102+0000","last_unstale":"2023-10-19T10:37:00.323709+0000","last_undegraded":"2023-10-19T10:37:00.323709+0000","last_fullsized":"2023-10-19T10:37:00.323709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323080+0000","last_change":"2023-10-19T10:36:55.108015+0000","last_active":"2023-10-19T10:37:00.323080+0000","last_peered":"2023-10-19T10:37:00.323080+0000","last_clean":"2023-10-19T10:37:00.323080+0000","last_became_active":"2023-10-19T10:36:55.107693+0000","last_became_peered":"2023-10-19T10:36:55.107693+0000","last_unstale":"2023-10-19T10:37:00.323080+0000","last_undegraded":"2023-10-19T10:37:00.323080+0000","last_fullsized":"2023-10-19T10:37:00.323080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097888+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:37:00.097888+0000","last_peered":"2023-10-19T10:37:00.097888+0000","last_clean":"2023-10-19T10:37:00.097888+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:37:00.097888+0000","last_undegraded":"2023-10-19T10:37:00.097888+0000","last_fullsized":"2023-10-19T10:37:00.097888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:31:56.734648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324508+0000","last_change":"2023-10-19T10:36:54.310814+0000","last_active":"2023-10-19T10:37:00.324508+0000","last_peered":"2023-10-19T10:37:00.324508+0000","last_clean":"2023-10-19T10:37:00.324508+0000","last_became_active":"2023-10-19T10:36:54.310517+0000","last_became_peered":"2023-10-19T10:36:54.310517+0000","last_unstale":"2023-10-19T10:37:00.324508+0000","last_undegraded":"2023-10-19T10:37:00.324508+0000","last_fullsized":"2023-10-19T10:37:00.324508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:37.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097777+0000","last_change":"2023-10-19T10:36:59.353687+0000","last_active":"2023-10-19T10:37:00.097777+0000","last_peered":"2023-10-19T10:37:00.097777+0000","last_clean":"2023-10-19T10:37:00.097777+0000","last_became_active":"2023-10-19T10:36:59.353297+0000","last_became_peered":"2023-10-19T10:36:59.353297+0000","last_unstale":"2023-10-19T10:37:00.097777+0000","last_undegraded":"2023-10-19T10:37:00.097777+0000","last_fullsized":"2023-10-19T10:37:00.097777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:05:36.096009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278152+0000","last_change":"2023-10-19T10:36:55.118741+0000","last_active":"2023-10-19T10:37:00.278152+0000","last_peered":"2023-10-19T10:37:00.278152+0000","last_clean":"2023-10-19T10:37:00.278152+0000","last_became_active":"2023-10-19T10:36:55.116465+0000","last_became_peered":"2023-10-19T10:36:55.116465+0000","last_unstale":"2023-10-19T10:37:00.278152+0000","last_undegraded":"2023-10-19T10:37:00.278152+0000","last_fullsized":"2023-10-19T10:37:00.278152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099045+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:37:00.099045+0000","last_peered":"2023-10-19T10:37:00.099045+0000","last_clean":"2023-10-19T10:37:00.099045+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:37:00.099045+0000","last_undegraded":"2023-10-19T10:37:00.099045+0000","last_fullsized":"2023-10-19T10:37:00.099045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:12.307342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097428+0000","last_change":"2023-10-19T10:36:54.306800+0000","last_active":"2023-10-19T10:37:00.097428+0000","last_peered":"2023-10-19T10:37:00.097428+0000","last_clean":"2023-10-19T10:37:00.097428+0000","last_became_active":"2023-10-19T10:36:54.306507+0000","last_became_peered":"2023-10-19T10:36:54.306507+0000","last_unstale":"2023-10-19T10:37:00.097428+0000","last_undegraded":"2023-10-19T10:37:00.097428+0000","last_fullsized":"2023-10-19T10:37:00.097428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:41.200123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323296+0000","last_change":"2023-10-19T10:36:59.352359+0000","last_active":"2023-10-19T10:37:00.323296+0000","last_peered":"2023-10-19T10:37:00.323296+0000","last_clean":"2023-10-19T10:37:00.323296+0000","last_became_active":"2023-10-19T10:36:59.351975+0000","last_became_peered":"2023-10-19T10:36:59.351975+0000","last_unstale":"2023-10-19T10:37:00.323296+0000","last_undegraded":"2023-10-19T10:37:00.323296+0000","last_fullsized":"2023-10-19T10:37:00.323296+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:39:36.789471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097181+0000","last_change":"2023-10-19T10:36:55.109753+0000","last_active":"2023-10-19T10:37:00.097181+0000","last_peered":"2023-10-19T10:37:00.097181+0000","last_clean":"2023-10-19T10:37:00.097181+0000","last_became_active":"2023-10-19T10:36:55.109424+0000","last_became_peered":"2023-10-19T10:36:55.109424+0000","last_unstale":"2023-10-19T10:37:00.097181+0000","last_undegraded":"2023-10-19T10:37:00.097181+0000","last_fullsized":"2023-10-19T10:37:00.097181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097504+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:37:00.097504+0000","last_peered":"2023-10-19T10:37:00.097504+0000","last_clean":"2023-10-19T10:37:00.097504+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:37:00.097504+0000","last_undegraded":"2023-10-19T10:37:00.097504+0000","last_fullsized":"2023-10-19T10:37:00.097504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:43.871421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324473+0000","last_change":"2023-10-19T10:36:54.305974+0000","last_active":"2023-10-19T10:37:00.324473+0000","last_peered":"2023-10-19T10:37:00.324473+0000","last_clean":"2023-10-19T10:37:00.324473+0000","last_became_active":"2023-10-19T10:36:54.305710+0000","last_became_peered":"2023-10-19T10:36:54.305710+0000","last_unstale":"2023-10-19T10:37:00.324473+0000","last_undegraded":"2023-10-19T10:37:00.324473+0000","last_fullsized":"2023-10-19T10:37:00.324473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277916+0000","last_change":"2023-10-19T10:36:59.353503+0000","last_active":"2023-10-19T10:37:00.277916+0000","last_peered":"2023-10-19T10:37:00.277916+0000","last_clean":"2023-10-19T10:37:00.277916+0000","last_became_active":"2023-10-19T10:36:59.352985+0000","last_became_peered":"2023-10-19T10:36:59.352985+0000","last_unstale":"2023-10-19T10:37:00.277916+0000","last_undegraded":"2023-10-19T10:37:00.277916+0000","last_fullsized":"2023-10-19T10:37:00.277916+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097415+0000","last_change":"2023-10-19T10:36:55.112577+0000","last_active":"2023-10-19T10:37:00.097415+0000","last_peered":"2023-10-19T10:37:00.097415+0000","last_clean":"2023-10-19T10:37:00.097415+0000","last_became_active":"2023-10-19T10:36:55.112302+0000","last_became_peered":"2023-10-19T10:36:55.112302+0000","last_unstale":"2023-10-19T10:37:00.097415+0000","last_undegraded":"2023-10-19T10:37:00.097415+0000","last_fullsized":"2023-10-19T10:37:00.097415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099330+0000","last_change":"2023-10-19T10:36:54.302990+0000","last_active":"2023-10-19T10:37:00.099330+0000","last_peered":"2023-10-19T10:37:00.099330+0000","last_clean":"2023-10-19T10:37:00.099330+0000","last_became_active":"2023-10-19T10:36:54.302553+0000","last_became_peered":"2023-10-19T10:36:54.302553+0000","last_unstale":"2023-10-19T10:37:00.099330+0000","last_undegraded":"2023-10-19T10:37:00.099330+0000","last_fullsized":"2023-10-19T10:37:00.099330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:07:06.761773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323861+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:37:00.323861+0000","last_peered":"2023-10-19T10:37:00.323861+0000","last_clean":"2023-10-19T10:37:00.323861+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:37:00.323861+0000","last_undegraded":"2023-10-19T10:37:00.323861+0000","last_fullsized":"2023-10-19T10:37:00.323861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:37.072096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277993+0000","last_change":"2023-10-19T10:36:59.351199+0000","last_active":"2023-10-19T10:37:00.277993+0000","last_peered":"2023-10-19T10:37:00.277993+0000","last_clean":"2023-10-19T10:37:00.277993+0000","last_became_active":"2023-10-19T10:36:59.350722+0000","last_became_peered":"2023-10-19T10:36:59.350722+0000","last_unstale":"2023-10-19T10:37:00.277993+0000","last_undegraded":"2023-10-19T10:37:00.277993+0000","last_fullsized":"2023-10-19T10:37:00.277993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097884+0000","last_change":"2023-10-19T10:36:55.108041+0000","last_active":"2023-10-19T10:37:00.097884+0000","last_peered":"2023-10-19T10:37:00.097884+0000","last_clean":"2023-10-19T10:37:00.097884+0000","last_became_active":"2023-10-19T10:36:55.107747+0000","last_became_peered":"2023-10-19T10:36:55.107747+0000","last_unstale":"2023-10-19T10:37:00.097884+0000","last_undegraded":"2023-10-19T10:37:00.097884+0000","last_fullsized":"2023-10-19T10:37:00.097884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096738+0000","last_change":"2023-10-19T10:36:54.307533+0000","last_active":"2023-10-19T10:37:00.096738+0000","last_peered":"2023-10-19T10:37:00.096738+0000","last_clean":"2023-10-19T10:37:00.096738+0000","last_became_active":"2023-10-19T10:36:54.307237+0000","last_became_peered":"2023-10-19T10:36:54.307237+0000","last_unstale":"2023-10-19T10:37:00.096738+0000","last_undegraded":"2023-10-19T10:37:00.096738+0000","last_fullsized":"2023-10-19T10:37:00.096738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:57.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322887+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:37:00.322887+0000","last_peered":"2023-10-19T10:37:00.322887+0000","last_clean":"2023-10-19T10:37:00.322887+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:37:00.322887+0000","last_undegraded":"2023-10-19T10:37:00.322887+0000","last_fullsized":"2023-10-19T10:37:00.322887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:35:30.635889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324367+0000","last_change":"2023-10-19T10:36:59.349089+0000","last_active":"2023-10-19T10:37:00.324367+0000","last_peered":"2023-10-19T10:37:00.324367+0000","last_clean":"2023-10-19T10:37:00.324367+0000","last_became_active":"2023-10-19T10:36:59.348703+0000","last_became_peered":"2023-10-19T10:36:59.348703+0000","last_unstale":"2023-10-19T10:37:00.324367+0000","last_undegraded":"2023-10-19T10:37:00.324367+0000","last_fullsized":"2023-10-19T10:37:00.324367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278271+0000","last_change":"2023-10-19T10:36:55.112250+0000","last_active":"2023-10-19T10:37:00.278271+0000","last_peered":"2023-10-19T10:37:00.278271+0000","last_clean":"2023-10-19T10:37:00.278271+0000","last_became_active":"2023-10-19T10:36:55.111996+0000","last_became_peered":"2023-10-19T10:36:55.111996+0000","last_unstale":"2023-10-19T10:37:00.278271+0000","last_undegraded":"2023-10-19T10:37:00.278271+0000","last_fullsized":"2023-10-19T10:37:00.278271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277550+0000","last_change":"2023-10-19T10:36:54.307098+0000","last_active":"2023-10-19T10:37:00.277550+0000","last_peered":"2023-10-19T10:37:00.277550+0000","last_clean":"2023-10-19T10:37:00.277550+0000","last_became_active":"2023-10-19T10:36:54.306769+0000","last_became_peered":"2023-10-19T10:36:54.306769+0000","last_unstale":"2023-10-19T10:37:00.277550+0000","last_undegraded":"2023-10-19T10:37:00.277550+0000","last_fullsized":"2023-10-19T10:37:00.277550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:13.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098430+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:37:00.098430+0000","last_peered":"2023-10-19T10:37:00.098430+0000","last_clean":"2023-10-19T10:37:00.098430+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:37:00.098430+0000","last_undegraded":"2023-10-19T10:37:00.098430+0000","last_fullsized":"2023-10-19T10:37:00.098430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324610+0000","last_change":"2023-10-19T10:36:59.357456+0000","last_active":"2023-10-19T10:37:00.324610+0000","last_peered":"2023-10-19T10:37:00.324610+0000","last_clean":"2023-10-19T10:37:00.324610+0000","last_became_active":"2023-10-19T10:36:59.357079+0000","last_became_peered":"2023-10-19T10:36:59.357079+0000","last_unstale":"2023-10-19T10:37:00.324610+0000","last_undegraded":"2023-10-19T10:37:00.324610+0000","last_fullsized":"2023-10-19T10:37:00.324610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:52.409495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323740+0000","last_change":"2023-10-19T10:36:55.113749+0000","last_active":"2023-10-19T10:37:00.323740+0000","last_peered":"2023-10-19T10:37:00.323740+0000","last_clean":"2023-10-19T10:37:00.323740+0000","last_became_active":"2023-10-19T10:36:55.113459+0000","last_became_peered":"2023-10-19T10:36:55.113459+0000","last_unstale":"2023-10-19T10:37:00.323740+0000","last_undegraded":"2023-10-19T10:37:00.323740+0000","last_fullsized":"2023-10-19T10:37:00.323740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:05:40.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323207+0000","last_change":"2023-10-19T10:36:54.309423+0000","last_active":"2023-10-19T10:37:00.323207+0000","last_peered":"2023-10-19T10:37:00.323207+0000","last_clean":"2023-10-19T10:37:00.323207+0000","last_became_active":"2023-10-19T10:36:54.309122+0000","last_became_peered":"2023-10-19T10:36:54.309122+0000","last_unstale":"2023-10-19T10:37:00.323207+0000","last_undegraded":"2023-10-19T10:37:00.323207+0000","last_fullsized":"2023-10-19T10:37:00.323207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:47.999447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324166+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:37:00.324166+0000","last_peered":"2023-10-19T10:37:00.324166+0000","last_clean":"2023-10-19T10:37:00.324166+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:37:00.324166+0000","last_undegraded":"2023-10-19T10:37:00.324166+0000","last_fullsized":"2023-10-19T10:37:00.324166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098587+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:37:00.098587+0000","last_peered":"2023-10-19T10:37:00.098587+0000","last_clean":"2023-10-19T10:37:00.098587+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:37:00.098587+0000","last_undegraded":"2023-10-19T10:37:00.098587+0000","last_fullsized":"2023-10-19T10:37:00.098587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:14.745066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098140+0000","last_change":"2023-10-19T10:36:59.359383+0000","last_active":"2023-10-19T10:37:00.098140+0000","last_peered":"2023-10-19T10:37:00.098140+0000","last_clean":"2023-10-19T10:37:00.098140+0000","last_became_active":"2023-10-19T10:36:59.358978+0000","last_became_peered":"2023-10-19T10:36:59.358978+0000","last_unstale":"2023-10-19T10:37:00.098140+0000","last_undegraded":"2023-10-19T10:37:00.098140+0000","last_fullsized":"2023-10-19T10:37:00.098140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097806+0000","last_change":"2023-10-19T10:36:55.111833+0000","last_active":"2023-10-19T10:37:00.097806+0000","last_peered":"2023-10-19T10:37:00.097806+0000","last_clean":"2023-10-19T10:37:00.097806+0000","last_became_active":"2023-10-19T10:36:55.111558+0000","last_became_peered":"2023-10-19T10:36:55.111558+0000","last_unstale":"2023-10-19T10:37:00.097806+0000","last_undegraded":"2023-10-19T10:37:00.097806+0000","last_fullsized":"2023-10-19T10:37:00.097806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323791+0000","last_change":"2023-10-19T10:36:54.304575+0000","last_active":"2023-10-19T10:37:00.323791+0000","last_peered":"2023-10-19T10:37:00.323791+0000","last_clean":"2023-10-19T10:37:00.323791+0000","last_became_active":"2023-10-19T10:36:54.304001+0000","last_became_peered":"2023-10-19T10:36:54.304001+0000","last_unstale":"2023-10-19T10:37:00.323791+0000","last_undegraded":"2023-10-19T10:37:00.323791+0000","last_fullsized":"2023-10-19T10:37:00.323791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099327+0000","last_change":"2023-10-19T10:36:59.347266+0000","last_active":"2023-10-19T10:37:00.099327+0000","last_peered":"2023-10-19T10:37:00.099327+0000","last_clean":"2023-10-19T10:37:00.099327+0000","last_became_active":"2023-10-19T10:36:59.346858+0000","last_became_peered":"2023-10-19T10:36:59.346858+0000","last_unstale":"2023-10-19T10:37:00.099327+0000","last_undegraded":"2023-10-19T10:37:00.099327+0000","last_fullsized":"2023-10-19T10:37:00.099327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:30.546568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099297+0000","last_change":"2023-10-19T10:36:55.107822+0000","last_active":"2023-10-19T10:37:00.099297+0000","last_peered":"2023-10-19T10:37:00.099297+0000","last_clean":"2023-10-19T10:37:00.099297+0000","last_became_active":"2023-10-19T10:36:55.107534+0000","last_became_peered":"2023-10-19T10:36:55.107534+0000","last_unstale":"2023-10-19T10:37:00.099297+0000","last_undegraded":"2023-10-19T10:37:00.099297+0000","last_fullsized":"2023-10-19T10:37:00.099297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:51:56.439816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098149+0000","last_change":"2023-10-19T10:36:54.306612+0000","last_active":"2023-10-19T10:37:00.098149+0000","last_peered":"2023-10-19T10:37:00.098149+0000","last_clean":"2023-10-19T10:37:00.098149+0000","last_became_active":"2023-10-19T10:36:54.306357+0000","last_became_peered":"2023-10-19T10:36:54.306357+0000","last_unstale":"2023-10-19T10:37:00.098149+0000","last_undegraded":"2023-10-19T10:37:00.098149+0000","last_fullsized":"2023-10-19T10:37:00.098149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:37:13.605874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324569+0000","last_change":"2023-10-19T10:36:59.352368+0000","last_active":"2023-10-19T10:37:00.324569+0000","last_peered":"2023-10-19T10:37:00.324569+0000","last_clean":"2023-10-19T10:37:00.324569+0000","last_became_active":"2023-10-19T10:36:59.352002+0000","last_became_peered":"2023-10-19T10:36:59.352002+0000","last_unstale":"2023-10-19T10:37:00.324569+0000","last_undegraded":"2023-10-19T10:37:00.324569+0000","last_fullsized":"2023-10-19T10:37:00.324569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:05.263646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099519+0000","last_change":"2023-10-19T10:36:55.110179+0000","last_active":"2023-10-19T10:37:00.099519+0000","last_peered":"2023-10-19T10:37:00.099519+0000","last_clean":"2023-10-19T10:37:00.099519+0000","last_became_active":"2023-10-19T10:36:55.109756+0000","last_became_peered":"2023-10-19T10:36:55.109756+0000","last_unstale":"2023-10-19T10:37:00.099519+0000","last_undegraded":"2023-10-19T10:37:00.099519+0000","last_fullsized":"2023-10-19T10:37:00.099519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:55:21.657235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323576+0000","last_change":"2023-10-19T10:36:54.306427+0000","last_active":"2023-10-19T10:37:00.323576+0000","last_peered":"2023-10-19T10:37:00.323576+0000","last_clean":"2023-10-19T10:37:00.323576+0000","last_became_active":"2023-10-19T10:36:54.306033+0000","last_became_peered":"2023-10-19T10:36:54.306033+0000","last_unstale":"2023-10-19T10:37:00.323576+0000","last_undegraded":"2023-10-19T10:37:00.323576+0000","last_fullsized":"2023-10-19T10:37:00.323576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:08:11.353560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097356+0000","last_change":"2023-10-19T10:36:59.353562+0000","last_active":"2023-10-19T10:37:00.097356+0000","last_peered":"2023-10-19T10:37:00.097356+0000","last_clean":"2023-10-19T10:37:00.097356+0000","last_became_active":"2023-10-19T10:36:59.353204+0000","last_became_peered":"2023-10-19T10:36:59.353204+0000","last_unstale":"2023-10-19T10:37:00.097356+0000","last_undegraded":"2023-10-19T10:37:00.097356+0000","last_fullsized":"2023-10-19T10:37:00.097356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099281+0000","last_change":"2023-10-19T10:36:55.115534+0000","last_active":"2023-10-19T10:37:00.099281+0000","last_peered":"2023-10-19T10:37:00.099281+0000","last_clean":"2023-10-19T10:37:00.099281+0000","last_became_active":"2023-10-19T10:36:55.114049+0000","last_became_peered":"2023-10-19T10:36:55.114049+0000","last_unstale":"2023-10-19T10:37:00.099281+0000","last_undegraded":"2023-10-19T10:37:00.099281+0000","last_fullsized":"2023-10-19T10:37:00.099281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:22:31.265043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323955+0000","last_change":"2023-10-19T10:36:54.310069+0000","last_active":"2023-10-19T10:37:00.323955+0000","last_peered":"2023-10-19T10:37:00.323955+0000","last_clean":"2023-10-19T10:37:00.323955+0000","last_became_active":"2023-10-19T10:36:54.309784+0000","last_became_peered":"2023-10-19T10:36:54.309784+0000","last_unstale":"2023-10-19T10:37:00.323955+0000","last_undegraded":"2023-10-19T10:37:00.323955+0000","last_fullsized":"2023-10-19T10:37:00.323955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323321+0000","last_change":"2023-10-19T10:36:59.351003+0000","last_active":"2023-10-19T10:37:00.323321+0000","last_peered":"2023-10-19T10:37:00.323321+0000","last_clean":"2023-10-19T10:37:00.323321+0000","last_became_active":"2023-10-19T10:36:59.349597+0000","last_became_peered":"2023-10-19T10:36:59.349597+0000","last_unstale":"2023-10-19T10:37:00.323321+0000","last_undegraded":"2023-10-19T10:37:00.323321+0000","last_fullsized":"2023-10-19T10:37:00.323321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:53:25.552100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277872+0000","last_change":"2023-10-19T10:36:55.118959+0000","last_active":"2023-10-19T10:37:00.277872+0000","last_peered":"2023-10-19T10:37:00.277872+0000","last_clean":"2023-10-19T10:37:00.277872+0000","last_became_active":"2023-10-19T10:36:55.116867+0000","last_became_peered":"2023-10-19T10:36:55.116867+0000","last_unstale":"2023-10-19T10:37:00.277872+0000","last_undegraded":"2023-10-19T10:37:00.277872+0000","last_fullsized":"2023-10-19T10:37:00.277872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324124+0000","last_change":"2023-10-19T10:36:54.305145+0000","last_active":"2023-10-19T10:37:00.324124+0000","last_peered":"2023-10-19T10:37:00.324124+0000","last_clean":"2023-10-19T10:37:00.324124+0000","last_became_active":"2023-10-19T10:36:54.304599+0000","last_became_peered":"2023-10-19T10:36:54.304599+0000","last_unstale":"2023-10-19T10:37:00.324124+0000","last_undegraded":"2023-10-19T10:37:00.324124+0000","last_fullsized":"2023-10-19T10:37:00.324124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323492+0000","last_change":"2023-10-19T10:36:59.348091+0000","last_active":"2023-10-19T10:37:00.323492+0000","last_peered":"2023-10-19T10:37:00.323492+0000","last_clean":"2023-10-19T10:37:00.323492+0000","last_became_active":"2023-10-19T10:36:59.346180+0000","last_became_peered":"2023-10-19T10:36:59.346180+0000","last_unstale":"2023-10-19T10:37:00.323492+0000","last_undegraded":"2023-10-19T10:37:00.323492+0000","last_fullsized":"2023-10-19T10:37:00.323492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:42:00.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323299+0000","last_change":"2023-10-19T10:36:55.114648+0000","last_active":"2023-10-19T10:37:00.323299+0000","last_peered":"2023-10-19T10:37:00.323299+0000","last_clean":"2023-10-19T10:37:00.323299+0000","last_became_active":"2023-10-19T10:36:55.114149+0000","last_became_peered":"2023-10-19T10:36:55.114149+0000","last_unstale":"2023-10-19T10:37:00.323299+0000","last_undegraded":"2023-10-19T10:37:00.323299+0000","last_fullsized":"2023-10-19T10:37:00.323299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:25:33.396338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098964+0000","last_change":"2023-10-19T10:36:54.309069+0000","last_active":"2023-10-19T10:37:00.098964+0000","last_peered":"2023-10-19T10:37:00.098964+0000","last_clean":"2023-10-19T10:37:00.098964+0000","last_became_active":"2023-10-19T10:36:54.308767+0000","last_became_peered":"2023-10-19T10:36:54.308767+0000","last_unstale":"2023-10-19T10:37:00.098964+0000","last_undegraded":"2023-10-19T10:37:00.098964+0000","last_fullsized":"2023-10-19T10:37:00.098964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323054+0000","last_change":"2023-10-19T10:36:59.349735+0000","last_active":"2023-10-19T10:37:00.323054+0000","last_peered":"2023-10-19T10:37:00.323054+0000","last_clean":"2023-10-19T10:37:00.323054+0000","last_became_active":"2023-10-19T10:36:59.349327+0000","last_became_peered":"2023-10-19T10:36:59.349327+0000","last_unstale":"2023-10-19T10:37:00.323054+0000","last_undegraded":"2023-10-19T10:37:00.323054+0000","last_fullsized":"2023-10-19T10:37:00.323054+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:51:57.082676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097759+0000","last_change":"2023-10-19T10:36:55.111295+0000","last_active":"2023-10-19T10:37:00.097759+0000","last_peered":"2023-10-19T10:37:00.097759+0000","last_clean":"2023-10-19T10:37:00.097759+0000","last_became_active":"2023-10-19T10:36:55.111016+0000","last_became_peered":"2023-10-19T10:36:55.111016+0000","last_unstale":"2023-10-19T10:37:00.097759+0000","last_undegraded":"2023-10-19T10:37:00.097759+0000","last_fullsized":"2023-10-19T10:37:00.097759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:01:05.121484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098592+0000","last_change":"2023-10-19T10:36:54.308800+0000","last_active":"2023-10-19T10:37:00.098592+0000","last_peered":"2023-10-19T10:37:00.098592+0000","last_clean":"2023-10-19T10:37:00.098592+0000","last_became_active":"2023-10-19T10:36:54.307919+0000","last_became_peered":"2023-10-19T10:36:54.307919+0000","last_unstale":"2023-10-19T10:37:00.098592+0000","last_undegraded":"2023-10-19T10:37:00.098592+0000","last_fullsized":"2023-10-19T10:37:00.098592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:00.862675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322763+0000","last_change":"2023-10-19T10:36:59.350698+0000","last_active":"2023-10-19T10:37:00.322763+0000","last_peered":"2023-10-19T10:37:00.322763+0000","last_clean":"2023-10-19T10:37:00.322763+0000","last_became_active":"2023-10-19T10:36:59.350331+0000","last_became_peered":"2023-10-19T10:36:59.350331+0000","last_unstale":"2023-10-19T10:37:00.322763+0000","last_undegraded":"2023-10-19T10:37:00.322763+0000","last_fullsized":"2023-10-19T10:37:00.322763+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:44.911501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323455+0000","last_change":"2023-10-19T10:36:55.110357+0000","last_active":"2023-10-19T10:37:00.323455+0000","last_peered":"2023-10-19T10:37:00.323455+0000","last_clean":"2023-10-19T10:37:00.323455+0000","last_became_active":"2023-10-19T10:36:55.110058+0000","last_became_peered":"2023-10-19T10:36:55.110058+0000","last_unstale":"2023-10-19T10:37:00.323455+0000","last_undegraded":"2023-10-19T10:37:00.323455+0000","last_fullsized":"2023-10-19T10:37:00.323455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:39.760467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098977+0000","last_change":"2023-10-19T10:36:54.305854+0000","last_active":"2023-10-19T10:37:00.098977+0000","last_peered":"2023-10-19T10:37:00.098977+0000","last_clean":"2023-10-19T10:37:00.098977+0000","last_became_active":"2023-10-19T10:36:54.305559+0000","last_became_peered":"2023-10-19T10:36:54.305559+0000","last_unstale":"2023-10-19T10:37:00.098977+0000","last_undegraded":"2023-10-19T10:37:00.098977+0000","last_fullsized":"2023-10-19T10:37:00.098977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097838+0000","last_change":"2023-10-19T10:36:59.359644+0000","last_active":"2023-10-19T10:37:00.097838+0000","last_peered":"2023-10-19T10:37:00.097838+0000","last_clean":"2023-10-19T10:37:00.097838+0000","last_became_active":"2023-10-19T10:36:59.359083+0000","last_became_peered":"2023-10-19T10:36:59.359083+0000","last_unstale":"2023-10-19T10:37:00.097838+0000","last_undegraded":"2023-10-19T10:37:00.097838+0000","last_fullsized":"2023-10-19T10:37:00.097838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324477+0000","last_change":"2023-10-19T10:36:59.349191+0000","last_active":"2023-10-19T10:37:00.324477+0000","last_peered":"2023-10-19T10:37:00.324477+0000","last_clean":"2023-10-19T10:37:00.324477+0000","last_became_active":"2023-10-19T10:36:59.348388+0000","last_became_peered":"2023-10-19T10:36:59.348388+0000","last_unstale":"2023-10-19T10:37:00.324477+0000","last_undegraded":"2023-10-19T10:37:00.324477+0000","last_fullsized":"2023-10-19T10:37:00.324477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323413+0000","last_change":"2023-10-19T10:36:54.311108+0000","last_active":"2023-10-19T10:37:00.323413+0000","last_peered":"2023-10-19T10:37:00.323413+0000","last_clean":"2023-10-19T10:37:00.323413+0000","last_became_active":"2023-10-19T10:36:54.310804+0000","last_became_peered":"2023-10-19T10:36:54.310804+0000","last_unstale":"2023-10-19T10:37:00.323413+0000","last_undegraded":"2023-10-19T10:37:00.323413+0000","last_fullsized":"2023-10-19T10:37:00.323413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:42:24.153777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324424+0000","last_change":"2023-10-19T10:36:55.108985+0000","last_active":"2023-10-19T10:37:00.324424+0000","last_peered":"2023-10-19T10:37:00.324424+0000","last_clean":"2023-10-19T10:37:00.324424+0000","last_became_active":"2023-10-19T10:36:55.108726+0000","last_became_peered":"2023-10-19T10:36:55.108726+0000","last_unstale":"2023-10-19T10:37:00.324424+0000","last_undegraded":"2023-10-19T10:37:00.324424+0000","last_fullsized":"2023-10-19T10:37:00.324424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:29.622437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324136+0000","last_change":"2023-10-19T10:36:59.352844+0000","last_active":"2023-10-19T10:37:00.324136+0000","last_peered":"2023-10-19T10:37:00.324136+0000","last_clean":"2023-10-19T10:37:00.324136+0000","last_became_active":"2023-10-19T10:36:59.352451+0000","last_became_peered":"2023-10-19T10:36:59.352451+0000","last_unstale":"2023-10-19T10:37:00.324136+0000","last_undegraded":"2023-10-19T10:37:00.324136+0000","last_fullsized":"2023-10-19T10:37:00.324136+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:21.477523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098487+0000","last_change":"2023-10-19T10:36:54.310021+0000","last_active":"2023-10-19T10:37:00.098487+0000","last_peered":"2023-10-19T10:37:00.098487+0000","last_clean":"2023-10-19T10:37:00.098487+0000","last_became_active":"2023-10-19T10:36:54.309686+0000","last_became_peered":"2023-10-19T10:36:54.309686+0000","last_unstale":"2023-10-19T10:37:00.098487+0000","last_undegraded":"2023-10-19T10:37:00.098487+0000","last_fullsized":"2023-10-19T10:37:00.098487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:25.891199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278441+0000","last_change":"2023-10-19T10:36:55.118465+0000","last_active":"2023-10-19T10:37:00.278441+0000","last_peered":"2023-10-19T10:37:00.278441+0000","last_clean":"2023-10-19T10:37:00.278441+0000","last_became_active":"2023-10-19T10:36:55.116202+0000","last_became_peered":"2023-10-19T10:36:55.116202+0000","last_unstale":"2023-10-19T10:37:00.278441+0000","last_undegraded":"2023-10-19T10:37:00.278441+0000","last_fullsized":"2023-10-19T10:37:00.278441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277600+0000","last_change":"2023-10-19T10:36:54.306522+0000","last_active":"2023-10-19T10:37:00.277600+0000","last_peered":"2023-10-19T10:37:00.277600+0000","last_clean":"2023-10-19T10:37:00.277600+0000","last_became_active":"2023-10-19T10:36:54.306147+0000","last_became_peered":"2023-10-19T10:36:54.306147+0000","last_unstale":"2023-10-19T10:37:00.277600+0000","last_undegraded":"2023-10-19T10:37:00.277600+0000","last_fullsized":"2023-10-19T10:37:00.277600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:38.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324371+0000","last_change":"2023-10-19T10:36:55.110979+0000","last_active":"2023-10-19T10:37:00.324371+0000","last_peered":"2023-10-19T10:37:00.324371+0000","last_clean":"2023-10-19T10:37:00.324371+0000","last_became_active":"2023-10-19T10:36:55.109937+0000","last_became_peered":"2023-10-19T10:36:55.109937+0000","last_unstale":"2023-10-19T10:37:00.324371+0000","last_undegraded":"2023-10-19T10:37:00.324371+0000","last_fullsized":"2023-10-19T10:37:00.324371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323485+0000","last_change":"2023-10-19T10:36:59.355501+0000","last_active":"2023-10-19T10:37:00.323485+0000","last_peered":"2023-10-19T10:37:00.323485+0000","last_clean":"2023-10-19T10:37:00.323485+0000","last_became_active":"2023-10-19T10:36:59.353627+0000","last_became_peered":"2023-10-19T10:36:59.353627+0000","last_unstale":"2023-10-19T10:37:00.323485+0000","last_undegraded":"2023-10-19T10:37:00.323485+0000","last_fullsized":"2023-10-19T10:37:00.323485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:30.933353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098816+0000","last_change":"2023-10-19T10:36:54.308041+0000","last_active":"2023-10-19T10:37:00.098816+0000","last_peered":"2023-10-19T10:37:00.098816+0000","last_clean":"2023-10-19T10:37:00.098816+0000","last_became_active":"2023-10-19T10:36:54.307752+0000","last_became_peered":"2023-10-19T10:36:54.307752+0000","last_unstale":"2023-10-19T10:37:00.098816+0000","last_undegraded":"2023-10-19T10:37:00.098816+0000","last_fullsized":"2023-10-19T10:37:00.098816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323551+0000","last_change":"2023-10-19T10:36:55.115058+0000","last_active":"2023-10-19T10:37:00.323551+0000","last_peered":"2023-10-19T10:37:00.323551+0000","last_clean":"2023-10-19T10:37:00.323551+0000","last_became_active":"2023-10-19T10:36:55.114729+0000","last_became_peered":"2023-10-19T10:36:55.114729+0000","last_unstale":"2023-10-19T10:37:00.323551+0000","last_undegraded":"2023-10-19T10:37:00.323551+0000","last_fullsized":"2023-10-19T10:37:00.323551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324110+0000","last_change":"2023-10-19T10:36:59.356762+0000","last_active":"2023-10-19T10:37:00.324110+0000","last_peered":"2023-10-19T10:37:00.324110+0000","last_clean":"2023-10-19T10:37:00.324110+0000","last_became_active":"2023-10-19T10:36:59.356422+0000","last_became_peered":"2023-10-19T10:36:59.356422+0000","last_unstale":"2023-10-19T10:37:00.324110+0000","last_undegraded":"2023-10-19T10:37:00.324110+0000","last_fullsized":"2023-10-19T10:37:00.324110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323028+0000","last_change":"2023-10-19T10:36:55.108618+0000","last_active":"2023-10-19T10:37:00.323028+0000","last_peered":"2023-10-19T10:37:00.323028+0000","last_clean":"2023-10-19T10:37:00.323028+0000","last_became_active":"2023-10-19T10:36:55.108364+0000","last_became_peered":"2023-10-19T10:36:55.108364+0000","last_unstale":"2023-10-19T10:37:00.323028+0000","last_undegraded":"2023-10-19T10:37:00.323028+0000","last_fullsized":"2023-10-19T10:37:00.323028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323784+0000","last_change":"2023-10-19T10:36:55.114936+0000","last_active":"2023-10-19T10:37:00.323784+0000","last_peered":"2023-10-19T10:37:00.323784+0000","last_clean":"2023-10-19T10:37:00.323784+0000","last_became_active":"2023-10-19T10:36:55.114673+0000","last_became_peered":"2023-10-19T10:36:55.114673+0000","last_unstale":"2023-10-19T10:37:00.323784+0000","last_undegraded":"2023-10-19T10:37:00.323784+0000","last_fullsized":"2023-10-19T10:37:00.323784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098152+0000","last_change":"2023-10-19T10:36:55.107446+0000","last_active":"2023-10-19T10:37:00.098152+0000","last_peered":"2023-10-19T10:37:00.098152+0000","last_clean":"2023-10-19T10:37:00.098152+0000","last_became_active":"2023-10-19T10:36:55.107185+0000","last_became_peered":"2023-10-19T10:36:55.107185+0000","last_unstale":"2023-10-19T10:37:00.098152+0000","last_undegraded":"2023-10-19T10:37:00.098152+0000","last_fullsized":"2023-10-19T10:37:00.098152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:36:13.224759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323701+0000","last_change":"2023-10-19T10:36:55.108586+0000","last_active":"2023-10-19T10:37:00.323701+0000","last_peered":"2023-10-19T10:37:00.323701+0000","last_clean":"2023-10-19T10:37:00.323701+0000","last_became_active":"2023-10-19T10:36:55.108308+0000","last_became_peered":"2023-10-19T10:36:55.108308+0000","last_unstale":"2023-10-19T10:37:00.323701+0000","last_undegraded":"2023-10-19T10:37:00.323701+0000","last_fullsized":"2023-10-19T10:37:00.323701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:09.378639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097485+0000","last_change":"2023-10-19T10:36:55.112395+0000","last_active":"2023-10-19T10:37:00.097485+0000","last_peered":"2023-10-19T10:37:00.097485+0000","last_clean":"2023-10-19T10:37:00.097485+0000","last_became_active":"2023-10-19T10:36:55.112101+0000","last_became_peered":"2023-10-19T10:36:55.112101+0000","last_unstale":"2023-10-19T10:37:00.097485+0000","last_undegraded":"2023-10-19T10:37:00.097485+0000","last_fullsized":"2023-10-19T10:37:00.097485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:55.114054+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:55.113749+0000","last_became_peered":"2023-10-19T10:36:55.113749+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097817+0000","last_change":"2023-10-19T10:36:55.109364+0000","last_active":"2023-10-19T10:37:00.097817+0000","last_peered":"2023-10-19T10:37:00.097817+0000","last_clean":"2023-10-19T10:37:00.097817+0000","last_became_active":"2023-10-19T10:36:55.109110+0000","last_became_peered":"2023-10-19T10:36:55.109110+0000","last_unstale":"2023-10-19T10:37:00.097817+0000","last_undegraded":"2023-10-19T10:37:00.097817+0000","last_fullsized":"2023-10-19T10:37:00.097817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097790+0000","last_change":"2023-10-19T10:36:55.111296+0000","last_active":"2023-10-19T10:37:00.097790+0000","last_peered":"2023-10-19T10:37:00.097790+0000","last_clean":"2023-10-19T10:37:00.097790+0000","last_became_active":"2023-10-19T10:36:55.111042+0000","last_became_peered":"2023-10-19T10:36:55.111042+0000","last_unstale":"2023-10-19T10:37:00.097790+0000","last_undegraded":"2023-10-19T10:37:00.097790+0000","last_fullsized":"2023-10-19T10:37:00.097790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098307+0000","last_change":"2023-10-19T10:36:55.117568+0000","last_active":"2023-10-19T10:37:00.098307+0000","last_peered":"2023-10-19T10:37:00.098307+0000","last_clean":"2023-10-19T10:37:00.098307+0000","last_became_active":"2023-10-19T10:36:55.117281+0000","last_became_peered":"2023-10-19T10:36:55.117281+0000","last_unstale":"2023-10-19T10:37:00.098307+0000","last_undegraded":"2023-10-19T10:37:00.098307+0000","last_fullsized":"2023-10-19T10:37:00.098307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323081+0000","last_change":"2023-10-19T10:36:55.114123+0000","last_active":"2023-10-19T10:37:00.323081+0000","last_peered":"2023-10-19T10:37:00.323081+0000","last_clean":"2023-10-19T10:37:00.323081+0000","last_became_active":"2023-10-19T10:36:55.113832+0000","last_became_peered":"2023-10-19T10:36:55.113832+0000","last_unstale":"2023-10-19T10:37:00.323081+0000","last_undegraded":"2023-10-19T10:37:00.323081+0000","last_fullsized":"2023-10-19T10:37:00.323081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277576+0000","last_change":"2023-10-19T10:36:55.115474+0000","last_active":"2023-10-19T10:37:00.277576+0000","last_peered":"2023-10-19T10:37:00.277576+0000","last_clean":"2023-10-19T10:37:00.277576+0000","last_became_active":"2023-10-19T10:36:55.115133+0000","last_became_peered":"2023-10-19T10:36:55.115133+0000","last_unstale":"2023-10-19T10:37:00.277576+0000","last_undegraded":"2023-10-19T10:37:00.277576+0000","last_fullsized":"2023-10-19T10:37:00.277576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098770+0000","last_change":"2023-10-19T10:36:55.117512+0000","last_active":"2023-10-19T10:37:00.098770+0000","last_peered":"2023-10-19T10:37:00.098770+0000","last_clean":"2023-10-19T10:37:00.098770+0000","last_became_active":"2023-10-19T10:36:55.117143+0000","last_became_peered":"2023-10-19T10:36:55.117143+0000","last_unstale":"2023-10-19T10:37:00.098770+0000","last_undegraded":"2023-10-19T10:37:00.098770+0000","last_fullsized":"2023-10-19T10:37:00.098770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:46.891573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099100+0000","last_change":"2023-10-19T10:36:55.106989+0000","last_active":"2023-10-19T10:37:00.099100+0000","last_peered":"2023-10-19T10:37:00.099100+0000","last_clean":"2023-10-19T10:37:00.099100+0000","last_became_active":"2023-10-19T10:36:55.106735+0000","last_became_peered":"2023-10-19T10:36:55.106735+0000","last_unstale":"2023-10-19T10:37:00.099100+0000","last_undegraded":"2023-10-19T10:37:00.099100+0000","last_fullsized":"2023-10-19T10:37:00.099100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:49:54.003217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324208+0000","last_change":"2023-10-19T10:36:55.107156+0000","last_active":"2023-10-19T10:37:00.324208+0000","last_peered":"2023-10-19T10:37:00.324208+0000","last_clean":"2023-10-19T10:37:00.324208+0000","last_became_active":"2023-10-19T10:36:55.106636+0000","last_became_peered":"2023-10-19T10:36:55.106636+0000","last_unstale":"2023-10-19T10:37:00.324208+0000","last_undegraded":"2023-10-19T10:37:00.324208+0000","last_fullsized":"2023-10-19T10:37:00.324208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:02:38.159925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098995+0000","last_change":"2023-10-19T10:36:55.102460+0000","last_active":"2023-10-19T10:37:00.098995+0000","last_peered":"2023-10-19T10:37:00.098995+0000","last_clean":"2023-10-19T10:37:00.098995+0000","last_became_active":"2023-10-19T10:36:55.102197+0000","last_became_peered":"2023-10-19T10:36:55.102197+0000","last_unstale":"2023-10-19T10:37:00.098995+0000","last_undegraded":"2023-10-19T10:37:00.098995+0000","last_fullsized":"2023-10-19T10:37:00.098995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:18:06.083802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278315+0000","last_change":"2023-10-19T10:36:55.118142+0000","last_active":"2023-10-19T10:37:00.278315+0000","last_peered":"2023-10-19T10:37:00.278315+0000","last_clean":"2023-10-19T10:37:00.278315+0000","last_became_active":"2023-10-19T10:36:55.116529+0000","last_became_peered":"2023-10-19T10:36:55.116529+0000","last_unstale":"2023-10-19T10:37:00.278315+0000","last_undegraded":"2023-10-19T10:37:00.278315+0000","last_fullsized":"2023-10-19T10:37:00.278315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097588+0000","last_change":"2023-10-19T10:36:55.110137+0000","last_active":"2023-10-19T10:37:00.097588+0000","last_peered":"2023-10-19T10:37:00.097588+0000","last_clean":"2023-10-19T10:37:00.097588+0000","last_became_active":"2023-10-19T10:36:55.109003+0000","last_became_peered":"2023-10-19T10:36:55.109003+0000","last_unstale":"2023-10-19T10:37:00.097588+0000","last_undegraded":"2023-10-19T10:37:00.097588+0000","last_fullsized":"2023-10-19T10:37:00.097588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:38.218180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098351+0000","last_change":"2023-10-19T10:36:55.115362+0000","last_active":"2023-10-19T10:37:00.098351+0000","last_peered":"2023-10-19T10:37:00.098351+0000","last_clean":"2023-10-19T10:37:00.098351+0000","last_became_active":"2023-10-19T10:36:55.114806+0000","last_became_peered":"2023-10-19T10:36:55.114806+0000","last_unstale":"2023-10-19T10:37:00.098351+0000","last_undegraded":"2023-10-19T10:37:00.098351+0000","last_fullsized":"2023-10-19T10:37:00.098351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323850+0000","last_change":"2023-10-19T10:36:55.106573+0000","last_active":"2023-10-19T10:37:00.323850+0000","last_peered":"2023-10-19T10:37:00.323850+0000","last_clean":"2023-10-19T10:37:00.323850+0000","last_became_active":"2023-10-19T10:36:55.106293+0000","last_became_peered":"2023-10-19T10:36:55.106293+0000","last_unstale":"2023-10-19T10:37:00.323850+0000","last_undegraded":"2023-10-19T10:37:00.323850+0000","last_fullsized":"2023-10-19T10:37:00.323850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323255+0000","last_change":"2023-10-19T10:36:55.116400+0000","last_active":"2023-10-19T10:37:00.323255+0000","last_peered":"2023-10-19T10:37:00.323255+0000","last_clean":"2023-10-19T10:37:00.323255+0000","last_became_active":"2023-10-19T10:36:55.116142+0000","last_became_peered":"2023-10-19T10:36:55.116142+0000","last_unstale":"2023-10-19T10:37:00.323255+0000","last_undegraded":"2023-10-19T10:37:00.323255+0000","last_fullsized":"2023-10-19T10:37:00.323255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:00:11.139684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323751+0000","last_change":"2023-10-19T10:36:55.102152+0000","last_active":"2023-10-19T10:37:00.323751+0000","last_peered":"2023-10-19T10:37:00.323751+0000","last_clean":"2023-10-19T10:37:00.323751+0000","last_became_active":"2023-10-19T10:36:55.101878+0000","last_became_peered":"2023-10-19T10:36:55.101878+0000","last_unstale":"2023-10-19T10:37:00.323751+0000","last_undegraded":"2023-10-19T10:37:00.323751+0000","last_fullsized":"2023-10-19T10:37:00.323751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:27:50.253900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099451+0000","last_change":"2023-10-19T10:36:55.111165+0000","last_active":"2023-10-19T10:37:00.099451+0000","last_peered":"2023-10-19T10:37:00.099451+0000","last_clean":"2023-10-19T10:37:00.099451+0000","last_became_active":"2023-10-19T10:36:55.110828+0000","last_became_peered":"2023-10-19T10:36:55.110828+0000","last_unstale":"2023-10-19T10:37:00.099451+0000","last_undegraded":"2023-10-19T10:37:00.099451+0000","last_fullsized":"2023-10-19T10:37:00.099451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:19:39.989618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324638+0000","last_change":"2023-10-19T10:36:55.110928+0000","last_active":"2023-10-19T10:37:00.324638+0000","last_peered":"2023-10-19T10:37:00.324638+0000","last_clean":"2023-10-19T10:37:00.324638+0000","last_became_active":"2023-10-19T10:36:55.110647+0000","last_became_peered":"2023-10-19T10:36:55.110647+0000","last_unstale":"2023-10-19T10:37:00.324638+0000","last_undegraded":"2023-10-19T10:37:00.324638+0000","last_fullsized":"2023-10-19T10:37:00.324638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:17.439266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278188+0000","last_change":"2023-10-19T10:36:55.118334+0000","last_active":"2023-10-19T10:37:00.278188+0000","last_peered":"2023-10-19T10:37:00.278188+0000","last_clean":"2023-10-19T10:37:00.278188+0000","last_became_active":"2023-10-19T10:36:55.116957+0000","last_became_peered":"2023-10-19T10:36:55.116957+0000","last_unstale":"2023-10-19T10:37:00.278188+0000","last_undegraded":"2023-10-19T10:37:00.278188+0000","last_fullsized":"2023-10-19T10:37:00.278188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323040+0000","last_change":"2023-10-19T10:36:55.110747+0000","last_active":"2023-10-19T10:37:00.323040+0000","last_peered":"2023-10-19T10:37:00.323040+0000","last_clean":"2023-10-19T10:37:00.323040+0000","last_became_active":"2023-10-19T10:36:55.110453+0000","last_became_peered":"2023-10-19T10:36:55.110453+0000","last_unstale":"2023-10-19T10:37:00.323040+0000","last_undegraded":"2023-10-19T10:37:00.323040+0000","last_fullsized":"2023-10-19T10:37:00.323040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097980+0000","last_change":"2023-10-19T10:36:55.111146+0000","last_active":"2023-10-19T10:37:00.097980+0000","last_peered":"2023-10-19T10:37:00.097980+0000","last_clean":"2023-10-19T10:37:00.097980+0000","last_became_active":"2023-10-19T10:36:55.110894+0000","last_became_peered":"2023-10-19T10:36:55.110894+0000","last_unstale":"2023-10-19T10:37:00.097980+0000","last_undegraded":"2023-10-19T10:37:00.097980+0000","last_fullsized":"2023-10-19T10:37:00.097980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097815+0000","last_change":"2023-10-19T10:36:55.109335+0000","last_active":"2023-10-19T10:37:00.097815+0000","last_peered":"2023-10-19T10:37:00.097815+0000","last_clean":"2023-10-19T10:37:00.097815+0000","last_became_active":"2023-10-19T10:36:55.109002+0000","last_became_peered":"2023-10-19T10:36:55.109002+0000","last_unstale":"2023-10-19T10:37:00.097815+0000","last_undegraded":"2023-10-19T10:37:00.097815+0000","last_fullsized":"2023-10-19T10:37:00.097815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:32:10.062310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278344+0000","last_change":"2023-10-19T10:36:55.119014+0000","last_active":"2023-10-19T10:37:00.278344+0000","last_peered":"2023-10-19T10:37:00.278344+0000","last_clean":"2023-10-19T10:37:00.278344+0000","last_became_active":"2023-10-19T10:36:55.118470+0000","last_became_peered":"2023-10-19T10:36:55.118470+0000","last_unstale":"2023-10-19T10:37:00.278344+0000","last_undegraded":"2023-10-19T10:37:00.278344+0000","last_fullsized":"2023-10-19T10:37:00.278344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323295+0000","last_change":"2023-10-19T10:36:55.114470+0000","last_active":"2023-10-19T10:37:00.323295+0000","last_peered":"2023-10-19T10:37:00.323295+0000","last_clean":"2023-10-19T10:37:00.323295+0000","last_became_active":"2023-10-19T10:36:55.113809+0000","last_became_peered":"2023-10-19T10:36:55.113809+0000","last_unstale":"2023-10-19T10:37:00.323295+0000","last_undegraded":"2023-10-19T10:37:00.323295+0000","last_fullsized":"2023-10-19T10:37:00.323295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:22:52.211277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099367+0000","last_change":"2023-10-19T10:36:55.113901+0000","last_active":"2023-10-19T10:37:00.099367+0000","last_peered":"2023-10-19T10:37:00.099367+0000","last_clean":"2023-10-19T10:37:00.099367+0000","last_became_active":"2023-10-19T10:36:55.113629+0000","last_became_peered":"2023-10-19T10:36:55.113629+0000","last_unstale":"2023-10-19T10:37:00.099367+0000","last_undegraded":"2023-10-19T10:37:00.099367+0000","last_fullsized":"2023-10-19T10:37:00.099367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:20:21.320100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324574+0000","last_change":"2023-10-19T10:36:55.112724+0000","last_active":"2023-10-19T10:37:00.324574+0000","last_peered":"2023-10-19T10:37:00.324574+0000","last_clean":"2023-10-19T10:37:00.324574+0000","last_became_active":"2023-10-19T10:36:55.112307+0000","last_became_peered":"2023-10-19T10:36:55.112307+0000","last_unstale":"2023-10-19T10:37:00.324574+0000","last_undegraded":"2023-10-19T10:37:00.324574+0000","last_fullsized":"2023-10-19T10:37:00.324574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:51.008519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324311+0000","last_change":"2023-10-19T10:36:55.107659+0000","last_active":"2023-10-19T10:37:00.324311+0000","last_peered":"2023-10-19T10:37:00.324311+0000","last_clean":"2023-10-19T10:37:00.324311+0000","last_became_active":"2023-10-19T10:36:55.107382+0000","last_became_peered":"2023-10-19T10:36:55.107382+0000","last_unstale":"2023-10-19T10:37:00.324311+0000","last_undegraded":"2023-10-19T10:37:00.324311+0000","last_fullsized":"2023-10-19T10:37:00.324311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":294912,"data_stored":119711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":888832,"data_stored":710571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:37:19.491 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-19T10:37:19.491 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-19T10:37:19.491 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-19T10:37:19.492 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-19T10:37:19.492 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-19T10:37:19.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.627+0000 7f087a6b1640 1 Processor -- start 2023-10-19T10:37:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.627+0000 7f087a6b1640 1 -- start start 2023-10-19T10:37:19.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f087a6b1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f087415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f087a6b1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f08740c27d0 0x7f0874070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f087a6b1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087415f050 con 0x7f087415b780 2023-10-19T10:37:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f087a6b1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08740cb990 con 0x7f08740c27d0 2023-10-19T10:37:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f087a6b1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087415e0c0 con 0x7f0874069f10 2023-10-19T10:37:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f08737fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f087415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f0878c4e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f08740c27d0 0x7f0874070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f08737fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f087415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43094/0 (socket says 172.21.15.50:43094) 2023-10-19T10:37:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f08737fe640 1 -- 172.21.15.50:0/1002199533 learned_addr learned my addr 172.21.15.50:0/1002199533 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.628+0000 7f0873fff640 1 --1- 172.21.15.50:0/1002199533 >> v1:172.21.15.183:6789/0 conn(0x7f087415b780 0x7f0874069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41836/0 (socket says 172.21.15.50:41836) 2023-10-19T10:37:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.629+0000 7f08737fe640 1 -- 172.21.15.50:0/1002199533 >> v2:172.21.15.183:3300/0 conn(0x7f08740c27d0 msgr2=0x7f0874070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.629+0000 7f08737fe640 1 --2- 172.21.15.50:0/1002199533 >> v2:172.21.15.183:3300/0 conn(0x7f08740c27d0 0x7f0874070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.629+0000 7f08737fe640 1 -- 172.21.15.50:0/1002199533 >> v1:172.21.15.183:6789/0 conn(0x7f087415b780 legacy=0x7f0874069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.629+0000 7f08737fe640 1 -- 172.21.15.50:0/1002199533 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740cc850 con 0x7f0874069f10 2023-10-19T10:37:19.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.629+0000 7f08737fe640 1 --2- 172.21.15.50:0/1002199533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f087415eb10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0864009870 tx=0x7f086402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75eecff14b727186 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f0872ffd640 1 -- 172.21.15.50:0/1002199533 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640317c0 con 0x7f0874069f10 2023-10-19T10:37:19.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f0872ffd640 1 -- 172.21.15.50:0/1002199533 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864031920 con 0x7f0874069f10 2023-10-19T10:37:19.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f0872ffd640 1 -- 172.21.15.50:0/1002199533 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864031cb0 con 0x7f0874069f10 2023-10-19T10:37:19.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 -- 172.21.15.50:0/1002199533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 msgr2=0x7f087415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1002199533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f087415eb10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0864009870 tx=0x7f086402b860 comp rx=0 tx=0).stop 2023-10-19T10:37:19.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 -- 172.21.15.50:0/1002199533 shutdown_connections 2023-10-19T10:37:19.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1002199533 >> v2:172.21.15.183:3300/0 conn(0x7f08740c27d0 0x7f0874070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1002199533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f087415eb10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 -- 172.21.15.50:0/1002199533 >> 172.21.15.50:0/1002199533 conn(0x7f08740cbbc0 msgr2=0x7f08740cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:19.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 -- 172.21.15.50:0/1002199533 shutdown_connections 2023-10-19T10:37:19.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.630+0000 7f087a6b1640 1 -- 172.21.15.50:0/1002199533 wait complete. 2023-10-19T10:37:19.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.631+0000 7f087a6b1640 1 Processor -- start 2023-10-19T10:37:19.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.631+0000 7f087a6b1640 1 -- start start 2023-10-19T10:37:19.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f087a6b1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f08741f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f087a6b1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 0x7f08740b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f087a6b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f087415b780 0x7f08740b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f087a6b1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740ce300 con 0x7f08740c27d0 2023-10-19T10:37:19.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f087a6b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740b2300 con 0x7f087415b780 2023-10-19T10:37:19.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f087a6b1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08741534f0 con 0x7f0874069f10 2023-10-19T10:37:19.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f08737fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 0x7f08740b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f08737fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 0x7f08740b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38824/0 (socket says 172.21.15.50:38824) 2023-10-19T10:37:19.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f08737fe640 1 -- 172.21.15.50:0/1652189890 learned_addr learned my addr 172.21.15.50:0/1652189890 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:19.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.632+0000 7f0878c4e640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f087415b780 0x7f08740b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f0873fff640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f08741f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f08737fe640 1 -- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 msgr2=0x7f08741f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f08737fe640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f08741f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f08737fe640 1 -- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f087415b780 msgr2=0x7f08740b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f08737fe640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f087415b780 0x7f08740b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f08737fe640 1 -- 172.21.15.50:0/1652189890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640094d0 con 0x7f08740c27d0 2023-10-19T10:37:19.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f08737fe640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 0x7f08740b49e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0864000c00 tx=0x7f08640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864032860 con 0x7f08740c27d0 2023-10-19T10:37:19.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08640329c0 con 0x7f08740c27d0 2023-10-19T10:37:19.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864032d70 con 0x7f08740c27d0 2023-10-19T10:37:19.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08740cda60 con 0x7f08740c27d0 2023-10-19T10:37:19.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.633+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740ce020 con 0x7f08740c27d0 2023-10-19T10:37:19.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.634+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874071c70 con 0x7f08740c27d0 2023-10-19T10:37:19.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.634+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f086404d3f0 con 0x7f08740c27d0 2023-10-19T10:37:19.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.635+0000 7f0870ff9640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f085406a050 0x7f085406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.635+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f08640aa500 con 0x7f08740c27d0 2023-10-19T10:37:19.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.635+0000 7f0873fff640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f085406a050 0x7f085406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.636+0000 7f0873fff640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f085406a050 0x7f085406c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f085c000f60 tx=0x7f085c000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.639+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f08640759b0 con 0x7f08740c27d0 2023-10-19T10:37:19.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.862+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0874073990 con 0x7f08740c27d0 2023-10-19T10:37:19.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.865+0000 7f0870ff9640 1 -- 172.21.15.50:0/1652189890 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169492 (secure 0 0 0) 0x7f0864078fd0 con 0x7f08740c27d0 2023-10-19T10:37:19.870 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f085406a050 msgr2=0x7f085406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f085406a050 0x7f085406c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f085c000f60 tx=0x7f085c000ed0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 msgr2=0x7f08740b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 0x7f08740b49e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0864000c00 tx=0x7f08640046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 shutdown_connections 2023-10-19T10:37:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f085406a050 0x7f085406c510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f087415b780 0x7f08740b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08740c27d0 0x7f08740b49e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 --2- 172.21.15.50:0/1652189890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0874069f10 0x7f08741f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 >> 172.21.15.50:0/1652189890 conn(0x7f08740cbbc0 msgr2=0x7f0874150c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:19.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 shutdown_connections 2023-10-19T10:37:19.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.868+0000 7f087a6b1640 1 -- 172.21.15.50:0/1652189890 wait complete. 2023-10-19T10:37:19.881 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":645958316},{"type":"v1","addr":"172.21.15.183:6833","nonce":645958316}]},"active_addr":"172.21.15.183:6833/645958316","active_change":"2023-10-19T10:36:34.451430+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":9400414}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":2060442600}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":192097151}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":3831343573}]}]} 2023-10-19T10:37:19.885 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-19T10:37:19.885 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-19T10:37:19.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:19.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.989+0000 7f3fe7836640 1 Processor -- start 2023-10-19T10:37:19.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.989+0000 7f3fe7836640 1 -- start start 2023-10-19T10:37:19.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe7836640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3fe015c570 0x7f3fe00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe7836640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe015c950 0x7f3fe00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe7836640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00cde60 con 0x7f3fe006b760 2023-10-19T10:37:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe7836640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe006aa30 con 0x7f3fe015c570 2023-10-19T10:37:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe7836640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006b2d0 con 0x7f3fe015c950 2023-10-19T10:37:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe55d2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3fe015c570 0x7f3fe00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe5dd3640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f3fe006b760 0x7f3fe00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41852/0 (socket says 172.21.15.50:41852) 2023-10-19T10:37:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe55d2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3fe015c570 0x7f3fe00d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:55704/0 (socket says 172.21.15.50:55704) 2023-10-19T10:37:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.990+0000 7f3fe5dd3640 1 -- 172.21.15.50:0/2044934969 learned_addr learned my addr 172.21.15.50:0/2044934969 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.991+0000 7f3fe4dd1640 1 --2- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe015c950 0x7f3fe00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.991+0000 7f3fe55d2640 1 -- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe015c950 msgr2=0x7f3fe00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:19.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.991+0000 7f3fe55d2640 1 --2- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe015c950 0x7f3fe00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:19.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.991+0000 7f3fe55d2640 1 -- 172.21.15.50:0/2044934969 >> v1:172.21.15.183:6789/0 conn(0x7f3fe006b760 legacy=0x7f3fe00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:19.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.991+0000 7f3fe55d2640 1 -- 172.21.15.50:0/2044934969 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00ce090 con 0x7f3fe015c570 2023-10-19T10:37:19.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.991+0000 7f3fe55d2640 1 --2- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c570 0x7f3fe00d0e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009790 tx=0x7f3fdc02b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595e92813a500034 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:19.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fd3fff640 1 -- 172.21.15.50:0/2044934969 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc031770 con 0x7f3fe015c570 2023-10-19T10:37:19.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fd3fff640 1 -- 172.21.15.50:0/2044934969 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0318d0 con 0x7f3fe015c570 2023-10-19T10:37:19.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fd3fff640 1 -- 172.21.15.50:0/2044934969 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc031c60 con 0x7f3fe015c570 2023-10-19T10:37:20.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 -- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c570 msgr2=0x7f3fe00d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 --2- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c570 0x7f3fe00d0e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009790 tx=0x7f3fdc02b810 comp rx=0 tx=0).stop 2023-10-19T10:37:20.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 -- 172.21.15.50:0/2044934969 shutdown_connections 2023-10-19T10:37:20.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 --2- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe015c950 0x7f3fe00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 --2- 172.21.15.50:0/2044934969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c570 0x7f3fe00d0e30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 -- 172.21.15.50:0/2044934969 >> 172.21.15.50:0/2044934969 conn(0x7f3fe0089ff0 msgr2=0x7f3fe009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 -- 172.21.15.50:0/2044934969 shutdown_connections 2023-10-19T10:37:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.992+0000 7f3fe7836640 1 -- 172.21.15.50:0/2044934969 wait complete. 2023-10-19T10:37:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.993+0000 7f3fe7836640 1 Processor -- start 2023-10-19T10:37:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.993+0000 7f3fe7836640 1 -- start start 2023-10-19T10:37:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe7836640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe006b760 0x7f3fe0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe7836640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 0x7f3fe015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe7836640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c950 0x7f3fe015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe7836640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c7ab0 con 0x7f3fe015c570 2023-10-19T10:37:20.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe7836640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe015fc40 con 0x7f3fe015c950 2023-10-19T10:37:20.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe7836640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c73f0 con 0x7f3fe006b760 2023-10-19T10:37:20.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe4dd1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 0x7f3fe015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe4dd1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 0x7f3fe015f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38834/0 (socket says 172.21.15.50:38834) 2023-10-19T10:37:20.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe4dd1640 1 -- 172.21.15.50:0/3929676108 learned_addr learned my addr 172.21.15.50:0/3929676108 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe55d2640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe006b760 0x7f3fe0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.994+0000 7f3fe5dd3640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c950 0x7f3fe015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe4dd1640 1 -- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe006b760 msgr2=0x7f3fe0153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe4dd1640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe006b760 0x7f3fe0153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe4dd1640 1 -- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c950 msgr2=0x7f3fe015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe4dd1640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c950 0x7f3fe015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe4dd1640 1 -- 172.21.15.50:0/3929676108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0093a0 con 0x7f3fe015c570 2023-10-19T10:37:20.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe5dd3640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c950 0x7f3fe015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:20.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe4dd1640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 0x7f3fe015f580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400d800 tx=0x7f3fd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400b6c0 con 0x7f3fe015c570 2023-10-19T10:37:20.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd400b820 con 0x7f3fe015c570 2023-10-19T10:37:20.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe00c76d0 con 0x7f3fe015c570 2023-10-19T10:37:20.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400bbb0 con 0x7f3fe015c570 2023-10-19T10:37:20.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.995+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe00c7ed0 con 0x7f3fe015c570 2023-10-19T10:37:20.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.996+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe00c8150 con 0x7f3fe015c570 2023-10-19T10:37:20.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.997+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3fd400bd10 con 0x7f3fe015c570 2023-10-19T10:37:20.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.997+0000 7f3fd1ffb640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fc406a050 0x7f3fc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.997+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3fd4088e10 con 0x7f3fe015c570 2023-10-19T10:37:20.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.997+0000 7f3fe55d2640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fc406a050 0x7f3fc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:19.998+0000 7f3fe55d2640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fc406a050 0x7f3fc406c510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc036000 tx=0x7f3fdc03d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.001+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3fd4054210 con 0x7f3fe015c570 2023-10-19T10:37:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.179+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fe0073990 con 0x7f3fe015c570 2023-10-19T10:37:20.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.180+0000 7f3fd1ffb640 1 -- 172.21.15.50:0/3929676108 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f3fd4057830 con 0x7f3fe015c570 2023-10-19T10:37:20.183 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:20.184 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fc406a050 msgr2=0x7f3fc406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fc406a050 0x7f3fc406c510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc036000 tx=0x7f3fdc03d040 comp rx=0 tx=0).stop 2023-10-19T10:37:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 msgr2=0x7f3fe015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 0x7f3fe015f580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400d800 tx=0x7f3fd400dcc0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 shutdown_connections 2023-10-19T10:37:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fc406a050 0x7f3fc406c510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fe015c950 0x7f3fe015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fe015c570 0x7f3fe015f580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 --2- 172.21.15.50:0/3929676108 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fe006b760 0x7f3fe0153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 >> 172.21.15.50:0/3929676108 conn(0x7f3fe0089ff0 msgr2=0x7f3fe00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 shutdown_connections 2023-10-19T10:37:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.182+0000 7f3fe7836640 1 -- 172.21.15.50:0/3929676108 wait complete. 2023-10-19T10:37:20.196 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-19T10:37:20.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:20.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.299+0000 7f64e9848640 1 Processor -- start 2023-10-19T10:37:20.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.299+0000 7f64e9848640 1 -- start start 2023-10-19T10:37:20.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e9848640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e415c3a0 0x7f64e40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e9848640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64e406eeb0 0x7f64e40cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e9848640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64e4069800 0x7f64e406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e9848640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e41611d0 con 0x7f64e4069800 2023-10-19T10:37:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e9848640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e41605c0 con 0x7f64e406eeb0 2023-10-19T10:37:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e9848640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e41608a0 con 0x7f64e415c3a0 2023-10-19T10:37:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64e4069800 0x7f64e406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64e4069800 0x7f64e406dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38844/0 (socket says 172.21.15.50:38844) 2023-10-19T10:37:20.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.300+0000 7f64e37fe640 1 -- 172.21.15.50:0/932362286 learned_addr learned my addr 172.21.15.50:0/932362286 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e27fc640 1 --2- 172.21.15.50:0/932362286 >> v2:172.21.15.183:3300/0 conn(0x7f64e406eeb0 0x7f64e40cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e2ffd640 1 --2- 172.21.15.50:0/932362286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e415c3a0 0x7f64e40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e37fe640 1 -- 172.21.15.50:0/932362286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e415c3a0 msgr2=0x7f64e40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e37fe640 1 --2- 172.21.15.50:0/932362286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e415c3a0 0x7f64e40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e37fe640 1 -- 172.21.15.50:0/932362286 >> v2:172.21.15.183:3300/0 conn(0x7f64e406eeb0 msgr2=0x7f64e40cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e37fe640 1 --2- 172.21.15.50:0/932362286 >> v2:172.21.15.183:3300/0 conn(0x7f64e406eeb0 0x7f64e40cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e37fe640 1 -- 172.21.15.50:0/932362286 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4160b80 con 0x7f64e4069800 2023-10-19T10:37:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.301+0000 7f64e37fe640 1 --2- 172.21.15.50:0/932362286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4069800 0x7f64e406dc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64e40663d0 tx=0x7f64d402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be750394eebafee server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e1ffb640 1 -- 172.21.15.50:0/932362286 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d402c9e0 con 0x7f64e4069800 2023-10-19T10:37:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e1ffb640 1 -- 172.21.15.50:0/932362286 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d402cb40 con 0x7f64e4069800 2023-10-19T10:37:20.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e1ffb640 1 -- 172.21.15.50:0/932362286 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40343f0 con 0x7f64e4069800 2023-10-19T10:37:20.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 -- 172.21.15.50:0/932362286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4069800 msgr2=0x7f64e406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 --2- 172.21.15.50:0/932362286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4069800 0x7f64e406dc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64e40663d0 tx=0x7f64d402be40 comp rx=0 tx=0).stop 2023-10-19T10:37:20.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 -- 172.21.15.50:0/932362286 shutdown_connections 2023-10-19T10:37:20.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 --2- 172.21.15.50:0/932362286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4069800 0x7f64e406dc00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 --2- 172.21.15.50:0/932362286 >> v2:172.21.15.183:3300/0 conn(0x7f64e406eeb0 0x7f64e40cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 --2- 172.21.15.50:0/932362286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e415c3a0 0x7f64e40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 -- 172.21.15.50:0/932362286 >> 172.21.15.50:0/932362286 conn(0x7f64e40cbc00 msgr2=0x7f64e40cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 -- 172.21.15.50:0/932362286 shutdown_connections 2023-10-19T10:37:20.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.302+0000 7f64e9848640 1 -- 172.21.15.50:0/932362286 wait complete. 2023-10-19T10:37:20.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.303+0000 7f64e9848640 1 Processor -- start 2023-10-19T10:37:20.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 -- start start 2023-10-19T10:37:20.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e4069800 0x7f64e41f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e406eeb0 0x7f64e415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 0x7f64e406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e415a850 con 0x7f64e415c3a0 2023-10-19T10:37:20.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e40c84c0 con 0x7f64e406eeb0 2023-10-19T10:37:20.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e9848640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e40c87a0 con 0x7f64e4069800 2023-10-19T10:37:20.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 0x7f64e406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 0x7f64e406dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38850/0 (socket says 172.21.15.50:38850) 2023-10-19T10:37:20.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.304+0000 7f64e37fe640 1 -- 172.21.15.50:0/4207939452 learned_addr learned my addr 172.21.15.50:0/4207939452 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e27fc640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e406eeb0 0x7f64e415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e2ffd640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e4069800 0x7f64e41f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e37fe640 1 -- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e4069800 msgr2=0x7f64e41f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e37fe640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e4069800 0x7f64e41f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e37fe640 1 -- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e406eeb0 msgr2=0x7f64e415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e37fe640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e406eeb0 0x7f64e415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e37fe640 1 -- 172.21.15.50:0/4207939452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4009b30 con 0x7f64e415c3a0 2023-10-19T10:37:20.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e37fe640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 0x7f64e406dc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64d4036040 tx=0x7f64d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4009510 con 0x7f64e415c3a0 2023-10-19T10:37:20.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d403e450 con 0x7f64e415c3a0 2023-10-19T10:37:20.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e40c8a80 con 0x7f64e415c3a0 2023-10-19T10:37:20.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d403e800 con 0x7f64e415c3a0 2023-10-19T10:37:20.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.305+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e40c9040 con 0x7f64e415c3a0 2023-10-19T10:37:20.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.306+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e40c2c90 con 0x7f64e415c3a0 2023-10-19T10:37:20.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.307+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f64d403e960 con 0x7f64e415c3a0 2023-10-19T10:37:20.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.307+0000 7f64dbfff640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.307+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f64d40ab510 con 0x7f64e415c3a0 2023-10-19T10:37:20.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.307+0000 7f64e2ffd640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.308+0000 7f64e2ffd640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64dc006fd0 tx=0x7f64dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.312+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f64e40c2c90 con 0x7f64e415c3a0 2023-10-19T10:37:20.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.488+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64e40c2c90 con 0x7f64e415c3a0 2023-10-19T10:37:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.489+0000 7f64dbfff640 1 -- 172.21.15.50:0/4207939452 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f64e40c2c90 con 0x7f64e415c3a0 2023-10-19T10:37:20.492 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:20.492 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:20.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 msgr2=0x7f64c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64dc006fd0 tx=0x7f64dc008040 comp rx=0 tx=0).stop 2023-10-19T10:37:20.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 msgr2=0x7f64e406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 0x7f64e406dc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64d4036040 tx=0x7f64d4004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:20.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 shutdown_connections 2023-10-19T10:37:20.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e415c3a0 0x7f64e406dc00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e406eeb0 0x7f64e415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 --2- 172.21.15.50:0/4207939452 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e4069800 0x7f64e41f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 >> 172.21.15.50:0/4207939452 conn(0x7f64e40cbc00 msgr2=0x7f64e40af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 shutdown_connections 2023-10-19T10:37:20.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.491+0000 7f64e9848640 1 -- 172.21.15.50:0/4207939452 wait complete. 2023-10-19T10:37:20.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-19T10:37:20.506 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-19T10:37:20.507 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-19T10:37:20.507 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-19T10:37:20.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-19T10:37:20.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-19T10:37:20.509 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-19T10:37:20.509 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-19T10:37:20.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.690+0000 7f47a05f0640 1 Processor -- start 2023-10-19T10:37:20.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.690+0000 7f47a05f0640 1 -- start start 2023-10-19T10:37:20.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.691+0000 7ff6ffd3e640 1 Processor -- start 2023-10-19T10:37:20.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.691+0000 7f005ffff640 1 Processor -- start 2023-10-19T10:37:20.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f47a05f0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 0x7f479816cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.691+0000 7f005ffff640 1 -- start start 2023-10-19T10:37:20.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7fa1db258640 1 Processor -- start 2023-10-19T10:37:20.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7ff545636640 1 Processor -- start 2023-10-19T10:37:20.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7ff545636640 1 -- start start 2023-10-19T10:37:20.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f47a05f0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f479806ff10 0x7f4798070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f47a05f0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980cc560 con 0x7f479816d6d0 2023-10-19T10:37:20.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f47a05f0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479816d450 con 0x7f479816cb30 2023-10-19T10:37:20.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f479e38c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 0x7f479816cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f479e38c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 0x7f479816cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38860/0 (socket says 172.21.15.50:38860) 2023-10-19T10:37:20.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f479e38c640 1 -- 172.21.15.50:0/3214757684 learned_addr learned my addr 172.21.15.50:0/3214757684 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7fa1db258640 1 -- start start 2023-10-19T10:37:20.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7fd43720f640 1 Processor -- start 2023-10-19T10:37:20.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.692+0000 7f479eb8d640 1 --2- 172.21.15.50:0/3214757684 >> v2:172.21.15.183:3300/0 conn(0x7f479806ff10 0x7f4798070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f47a05f0640 1 -- 172.21.15.50:0/3214757684 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479806f670 con 0x7f479806ff10 2023-10-19T10:37:20.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff545636640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400d1570 0x7ff5400b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.691+0000 7ff6ffd3e640 1 -- start start 2023-10-19T10:37:20.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff6ffd3e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff6f80c82c0 0x7ff6f806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff6ffd3e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff6f806d460 0x7ff6f806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff6ffd3e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f8071610 con 0x7ff6f806d9b0 2023-10-19T10:37:20.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff6ffd3e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80ca1d0 con 0x7ff6f80c82c0 2023-10-19T10:37:20.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff6ffd3e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80ca4b0 con 0x7ff6f806d460 2023-10-19T10:37:20.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f005ffff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00580eac80 0x7f0058047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f005ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f0058006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f005ffff640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0058037b50 con 0x7f0058006880 2023-10-19T10:37:20.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f005ffff640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0058006d10 con 0x7f00580eac80 2023-10-19T10:37:20.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f005ffff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005804f710 con 0x7f0058005ec0 2023-10-19T10:37:20.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff545636640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff5400b1830 0x7ff5400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff545636640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff5400cc8c0 0x7ff54015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff545636640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400b43b0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff545636640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400d1e90 con 0x7ff5400b1830 2023-10-19T10:37:20.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff545636640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400cd1c0 con 0x7ff5400d1570 2023-10-19T10:37:20.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff53ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff5400cc8c0 0x7ff54015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff53ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff5400cc8c0 0x7ff54015ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38876/0 (socket says 172.21.15.50:38876) 2023-10-19T10:37:20.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff53ffff640 1 -- 172.21.15.50:0/2683337549 learned_addr learned my addr 172.21.15.50:0/2683337549 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f405d42f640 1 Processor -- start 2023-10-19T10:37:20.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f479d38a640 1 -- 172.21.15.50:0/3214757684 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256728837 0 0) 0x7f47980cc560 con 0x7f479816d6d0 2023-10-19T10:37:20.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f479d38a640 1 -- 172.21.15.50:0/3214757684 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4784005730 con 0x7f479816d6d0 2023-10-19T10:37:20.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff53effd640 1 --2- 172.21.15.50:0/2683337549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400d1570 0x7ff5400b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f479eb8d640 1 -- 172.21.15.50:0/3214757684 >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 msgr2=0x7f479816cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7ff6fe2db640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff6f806d460 0x7ff6f806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff6fe2db640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff6f806d460 0x7ff6f806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:55744/0 (socket says 172.21.15.50:55744) 2023-10-19T10:37:20.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff6fe2db640 1 -- 172.21.15.50:0/2122610678 learned_addr learned my addr 172.21.15.50:0/2122610678 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f479eb8d640 1 --2- 172.21.15.50:0/3214757684 >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 0x7f479816cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f479eb8d640 1 -- 172.21.15.50:0/3214757684 >> v1:172.21.15.50:6789/0 conn(0x7f479816d6d0 legacy=0x7f479806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f479eb8d640 1 -- 172.21.15.50:0/3214757684 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479806f950 con 0x7f479806ff10 2023-10-19T10:37:20.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f005f7fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0058006880 0x7f00580eb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41866/0 (socket says 172.21.15.50:41866) 2023-10-19T10:37:20.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f005f7fe640 1 -- 172.21.15.50:0/3533734532 learned_addr learned my addr 172.21.15.50:0/3533734532 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:20.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7fd43720f640 1 -- start start 2023-10-19T10:37:20.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fd43720f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd430160740 0x7fd4300ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fd43720f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4300af210 0x7fd4300c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fd43720f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4300ad6c0 con 0x7fd43006a910 2023-10-19T10:37:20.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fd43720f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4300af000 con 0x7fd430160740 2023-10-19T10:37:20.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fd43720f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd430160b20 con 0x7fd4300af210 2023-10-19T10:37:20.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fd436a0e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd43006a910 0x7fd4300cda60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33530/0 (socket says 172.21.15.50:33530) 2023-10-19T10:37:20.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd436a0e640 1 -- 172.21.15.50:0/3786180471 learned_addr learned my addr 172.21.15.50:0/3786180471 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:20.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.693+0000 7f405d42f640 1 -- start start 2023-10-19T10:37:20.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1db258640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1d415aec0 0x7fa1d40ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1db258640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa1d4161800 0x7fa1d40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1db258640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d4161e90 con 0x7fa1d40cadf0 2023-10-19T10:37:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1db258640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d4161c30 con 0x7fa1d4161800 2023-10-19T10:37:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1db258640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d41653d0 con 0x7fa1d415aec0 2023-10-19T10:37:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1d97f5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa1d4161800 0x7fa1d40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7fa1d97f5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa1d4161800 0x7fa1d40c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38884/0 (socket says 172.21.15.50:38884) 2023-10-19T10:37:20.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fa1d97f5640 1 -- 172.21.15.50:0/2952390330 learned_addr learned my addr 172.21.15.50:0/2952390330 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff6fe2db640 1 -- 172.21.15.50:0/2122610678 >> v2:172.21.15.50:3300/0 conn(0x7ff6f80c82c0 msgr2=0x7ff6f806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff6fe2db640 1 --2- 172.21.15.50:0/2122610678 >> v2:172.21.15.50:3300/0 conn(0x7ff6f80c82c0 0x7ff6f806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff6fe2db640 1 -- 172.21.15.50:0/2122610678 >> v1:172.21.15.50:6789/0 conn(0x7ff6f806d9b0 legacy=0x7ff6f80c86a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-19T10:37:20.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff6fe2db640 1 -- 172.21.15.50:0/2122610678 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8071050 con 0x7ff6f806d460 2023-10-19T10:37:20.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f005e7fc640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f0058006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7f005effd640 1 --2- 172.21.15.50:0/3533734532 >> v2:172.21.15.183:3300/0 conn(0x7f00580eac80 0x7f0058047bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005dffb640 1 -- 172.21.15.50:0/3533734532 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697849233 0 0) 0x7f0058037b50 con 0x7f0058006880 2023-10-19T10:37:20.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff53ffff640 1 -- 172.21.15.50:0/2683337549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400d1570 msgr2=0x7ff5400b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff53ffff640 1 --2- 172.21.15.50:0/2683337549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400d1570 0x7ff5400b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff53ffff640 1 -- 172.21.15.50:0/2683337549 >> v2:172.21.15.183:3300/0 conn(0x7ff5400b1830 msgr2=0x7ff5400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff53ffff640 1 --2- 172.21.15.50:0/2683337549 >> v2:172.21.15.183:3300/0 conn(0x7ff5400b1830 0x7ff5400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.694+0000 7ff53ffff640 1 -- 172.21.15.50:0/2683337549 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400cd4a0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd43620d640 1 --2- 172.21.15.50:0/3786180471 >> v2:172.21.15.183:3300/0 conn(0x7fd430160740 0x7fd4300ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005dffb640 1 -- 172.21.15.50:0/3533734532 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0048003520 con 0x7f0058006880 2023-10-19T10:37:20.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f405d42f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f405004fc40 0x7f4050006ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f405d42f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4050048170 0x7f4050038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f405d42f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4050037dc0 con 0x7f40500e76b0 2023-10-19T10:37:20.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f479e38c640 1 --2- 172.21.15.50:0/3214757684 >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 0x7f479816cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:20.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7ff53ffff640 1 --2- 172.21.15.50:0/2683337549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff54015ff00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5400b4dc0 tx=0x7ff53002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff74b65e912b894 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd435a0c640 1 --2- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4300af210 0x7fd4300c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f405d42f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4050037bd0 con 0x7f4050048170 2023-10-19T10:37:20.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005effd640 1 -- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 msgr2=0x7f0058006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd43620d640 1 -- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4300af210 msgr2=0x7fd4300c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd43620d640 1 --2- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4300af210 0x7fd4300c3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd43620d640 1 -- 172.21.15.50:0/3786180471 >> v1:172.21.15.50:6789/0 conn(0x7fd43006a910 legacy=0x7fd4300cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005effd640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f0058006340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f405d42f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4050007090 con 0x7f405004fc40 2023-10-19T10:37:20.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005effd640 1 -- 172.21.15.50:0/3533734532 >> v1:172.21.15.183:6789/0 conn(0x7f0058006880 legacy=0x7f00580eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005effd640 1 -- 172.21.15.50:0/3533734532 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005804f150 con 0x7f00580eac80 2023-10-19T10:37:20.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f005e7fc640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f0058006340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:20.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f4681899640 1 Processor -- start 2023-10-19T10:37:20.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f4681899640 1 -- start start 2023-10-19T10:37:20.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f479eb8d640 1 --2- 172.21.15.50:0/3214757684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479806ff10 0x7f4798070330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47980cf030 tx=0x7f478802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9814dc1b22ac19fe server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7ff6fe2db640 1 --2- 172.21.15.50:0/2122610678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 0x7ff6f806bce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806f260 tx=0x7ff6f002e090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1585a94a1e44251e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f4057fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4050048170 0x7f4050038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f4056ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f40500e76b0 0x7f405004f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33534/0 (socket says 172.21.15.50:33534) 2023-10-19T10:37:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f4056ffd640 1 -- 172.21.15.50:0/2723222343 learned_addr learned my addr 172.21.15.50:0/2723222343 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:20.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7f40567fc640 1 --2- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f405004fc40 0x7f4050006ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7ff6fcad8640 1 -- 172.21.15.50:0/2122610678 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f002ee40 con 0x7ff6f806d460 2023-10-19T10:37:20.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7ff6fcad8640 1 -- 172.21.15.50:0/2122610678 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0032070 con 0x7ff6f806d460 2023-10-19T10:37:20.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7ff6fcad8640 1 -- 172.21.15.50:0/2122610678 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0038600 con 0x7ff6f806d460 2023-10-19T10:37:20.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f479d38a640 1 -- 172.21.15.50:0/3214757684 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788031640 con 0x7f479806ff10 2023-10-19T10:37:20.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f479d38a640 1 -- 172.21.15.50:0/3214757684 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47880317a0 con 0x7f479806ff10 2023-10-19T10:37:20.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f479d38a640 1 -- 172.21.15.50:0/3214757684 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788031b50 con 0x7f479806ff10 2023-10-19T10:37:20.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fa1d8ff4640 1 --2- 172.21.15.50:0/2952390330 >> v2:172.21.15.183:3300/0 conn(0x7fa1d415aec0 0x7fa1d40ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.695+0000 7fd43620d640 1 -- 172.21.15.50:0/3786180471 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4300ad960 con 0x7fd430160740 2023-10-19T10:37:20.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7fa1d97f5640 1 -- 172.21.15.50:0/2952390330 >> v2:172.21.15.183:3300/0 conn(0x7fa1d415aec0 msgr2=0x7fa1d40ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7fa1d97f5640 1 --2- 172.21.15.50:0/2952390330 >> v2:172.21.15.183:3300/0 conn(0x7fa1d415aec0 0x7fa1d40ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7fa1d97f5640 1 -- 172.21.15.50:0/2952390330 >> v1:172.21.15.50:6789/0 conn(0x7fa1d40cadf0 legacy=0x7fa1d41610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f4055ffb640 1 -- 172.21.15.50:0/2723222343 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824242550 0 0) 0x7f4050037dc0 con 0x7f40500e76b0 2023-10-19T10:37:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f4057fff640 1 -- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f405004fc40 msgr2=0x7f4050006ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f4057fff640 1 --2- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f405004fc40 0x7f4050006ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f4057fff640 1 -- 172.21.15.50:0/2723222343 >> v1:172.21.15.50:6789/0 conn(0x7f40500e76b0 legacy=0x7f405004f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7f4057fff640 1 -- 172.21.15.50:0/2723222343 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050007370 con 0x7f4050048170 2023-10-19T10:37:20.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7fa1d97f5640 1 -- 172.21.15.50:0/2952390330 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4164610 con 0x7fa1d4161800 2023-10-19T10:37:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7ff53dffb640 1 -- 172.21.15.50:0/2683337549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530031880 con 0x7ff5400cc8c0 2023-10-19T10:37:20.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.696+0000 7fa1d8ff4640 1 --2- 172.21.15.50:0/2952390330 >> v2:172.21.15.183:3300/0 conn(0x7fa1d415aec0 0x7fa1d40ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:20.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff53dffb640 1 -- 172.21.15.50:0/2683337549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5300319e0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff53dffb640 1 -- 172.21.15.50:0/2683337549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530031d70 con 0x7ff5400cc8c0 2023-10-19T10:37:20.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f4681899640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46740e11e0 0x7f4674008a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/2122610678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 msgr2=0x7ff6f806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/2122610678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 0x7ff6f806bce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806f260 tx=0x7ff6f002e090 comp rx=0 tx=0).stop 2023-10-19T10:37:20.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f005effd640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 0x7f0058047bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00500097a0 tx=0x7f005002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7345ebad99d1a723 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/2122610678 shutdown_connections 2023-10-19T10:37:20.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7fd43620d640 1 --2- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd430160740 0x7fd4300ae720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097d0 tx=0x7fd42802b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd58489ce8b12a63 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7fd43520b640 1 -- 172.21.15.50:0/3786180471 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428031800 con 0x7fd430160740 2023-10-19T10:37:20.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7fd43520b640 1 -- 172.21.15.50:0/3786180471 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd428031960 con 0x7fd430160740 2023-10-19T10:37:20.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7fd43520b640 1 -- 172.21.15.50:0/3786180471 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428031cf0 con 0x7fd430160740 2023-10-19T10:37:20.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/2122610678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 0x7ff6f806bce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/2122610678 >> v2:172.21.15.50:3300/0 conn(0x7ff6f80c82c0 0x7ff6f806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/2122610678 >> 172.21.15.50:0/2122610678 conn(0x7ff6f80cdc70 msgr2=0x7ff6f80ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f005dffb640 1 -- 172.21.15.50:0/3533734532 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050030800 con 0x7f00580eac80 2023-10-19T10:37:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f4681899640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f467403b020 0x7f4674008f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f467affd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46740e11e0 0x7f4674008a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f467affd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46740e11e0 0x7f4674008a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:55804/0 (socket says 172.21.15.50:55804) 2023-10-19T10:37:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f467affd640 1 -- 172.21.15.50:0/2907325834 learned_addr learned my addr 172.21.15.50:0/2907325834 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467403b600 con 0x7f46740e15c0 2023-10-19T10:37:20.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f47a05f0640 1 -- 172.21.15.50:0/3214757684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479806ff10 msgr2=0x7f4798070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f47a05f0640 1 --2- 172.21.15.50:0/3214757684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479806ff10 0x7f4798070330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47980cf030 tx=0x7f478802b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4674050430 con 0x7f467403b020 2023-10-19T10:37:20.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4674050710 con 0x7f46740e11e0 2023-10-19T10:37:20.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f005dffb640 1 -- 172.21.15.50:0/3533734532 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050030960 con 0x7f00580eac80 2023-10-19T10:37:20.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.697+0000 7f005dffb640 1 -- 172.21.15.50:0/3533734532 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050030cf0 con 0x7f00580eac80 2023-10-19T10:37:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f47a05f0640 1 -- 172.21.15.50:0/3214757684 shutdown_connections 2023-10-19T10:37:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f47a05f0640 1 --2- 172.21.15.50:0/3214757684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479806ff10 0x7f4798070330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4057fff640 1 --2- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050048170 0x7f4050038270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f405802c740 tx=0x7f40580677b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6eae502b319d6c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f47a05f0640 1 --2- 172.21.15.50:0/3214757684 >> v2:172.21.15.50:3300/0 conn(0x7f479816cb30 0x7f479816cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fa1d97f5640 1 --2- 172.21.15.50:0/2952390330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4161800 0x7fa1d40c8660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1d406d950 tx=0x7fa1cc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2124c14a638962af server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f47a05f0640 1 -- 172.21.15.50:0/3214757684 >> 172.21.15.50:0/3214757684 conn(0x7f47980cdfd0 msgr2=0x7f47980ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 -- 172.21.15.50:0/2683337549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 msgr2=0x7ff54015ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 --2- 172.21.15.50:0/2683337549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff54015ff00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5400b4dc0 tx=0x7ff53002b920 comp rx=0 tx=0).stop 2023-10-19T10:37:20.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 -- 172.21.15.50:0/2683337549 shutdown_connections 2023-10-19T10:37:20.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 --2- 172.21.15.50:0/2683337549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff54015ff00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4055ffb640 1 -- 172.21.15.50:0/2723222343 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580a5dd0 con 0x7f4050048170 2023-10-19T10:37:20.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4055ffb640 1 -- 172.21.15.50:0/2723222343 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40580ad440 con 0x7f4050048170 2023-10-19T10:37:20.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4055ffb640 1 -- 172.21.15.50:0/2723222343 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580ad870 con 0x7f4050048170 2023-10-19T10:37:20.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382985354 0 0) 0x7f467403b600 con 0x7f46740e15c0 2023-10-19T10:37:20.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4664003520 con 0x7f46740e15c0 2023-10-19T10:37:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 --2- 172.21.15.50:0/2683337549 >> v2:172.21.15.183:3300/0 conn(0x7ff5400b1830 0x7ff5400d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/2122610678 shutdown_connections 2023-10-19T10:37:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 --2- 172.21.15.50:0/2683337549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400d1570 0x7ff5400b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 -- 172.21.15.50:0/2683337549 >> 172.21.15.50:0/2683337549 conn(0x7ff540070f30 msgr2=0x7ff540071340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fa1d37fe640 1 -- 172.21.15.50:0/2952390330 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004230 con 0x7fa1d4161800 2023-10-19T10:37:20.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 -- 172.21.15.50:0/2683337549 shutdown_connections 2023-10-19T10:37:20.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f467bfff640 1 --2- 172.21.15.50:0/2907325834 >> v2:172.21.15.50:3300/0 conn(0x7f467403b020 0x7f4674008f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fa1d37fe640 1 -- 172.21.15.50:0/2952390330 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc02e050 con 0x7fa1d4161800 2023-10-19T10:37:20.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fa1d37fe640 1 -- 172.21.15.50:0/2952390330 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc003610 con 0x7fa1d4161800 2023-10-19T10:37:20.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff545636640 1 -- 172.21.15.50:0/2683337549 wait complete. 2023-10-19T10:37:20.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 -- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd430160740 msgr2=0x7fd4300ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 --2- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd430160740 0x7fd4300ae720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097d0 tx=0x7fd42802b8a0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 -- 172.21.15.50:0/3786180471 shutdown_connections 2023-10-19T10:37:20.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 --2- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4300af210 0x7fd4300c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 --2- 172.21.15.50:0/3786180471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd430160740 0x7fd4300ae720 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097d0 tx=0x7fd42802b8a0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 -- 172.21.15.50:0/3786180471 >> 172.21.15.50:0/3786180471 conn(0x7fd4300cff20 msgr2=0x7fd4300d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 -- 172.21.15.50:0/3786180471 shutdown_connections 2023-10-19T10:37:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7fd43720f640 1 -- 172.21.15.50:0/3786180471 wait complete. 2023-10-19T10:37:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f005ffff640 1 -- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 msgr2=0x7f0058047bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f005ffff640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 0x7f0058047bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00500097a0 tx=0x7f005002b940 comp rx=0 tx=0).stop 2023-10-19T10:37:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f005ffff640 1 -- 172.21.15.50:0/3533734532 shutdown_connections 2023-10-19T10:37:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f005ffff640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f0058006340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f005ffff640 1 --2- 172.21.15.50:0/3533734532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 0x7f0058047bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f005ffff640 1 -- 172.21.15.50:0/3533734532 >> 172.21.15.50:0/3533734532 conn(0x7f0058009fe0 msgr2=0x7f005800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728844639 0 0) 0x7f4664003520 con 0x7f46740e15c0 2023-10-19T10:37:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 >> v2:172.21.15.183:3300/0 conn(0x7f46740e11e0 msgr2=0x7f4674008a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 --2- 172.21.15.50:0/2907325834 >> v2:172.21.15.183:3300/0 conn(0x7f46740e11e0 0x7f4674008a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 >> v2:172.21.15.50:3300/0 conn(0x7f467403b020 msgr2=0x7f4674008f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 --2- 172.21.15.50:0/2907325834 >> v2:172.21.15.50:3300/0 conn(0x7f467403b020 0x7f4674008f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740509f0 con 0x7f46740e15c0 2023-10-19T10:37:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f4670001670 con 0x7f46740e15c0 2023-10-19T10:37:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 >> v1:172.21.15.50:6789/0 conn(0x7f46740e15c0 legacy=0x7f467403a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f4664003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46640047c0 0x7f4664004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664009970 0x7f46640051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4674050430 con 0x7f4664003990 2023-10-19T10:37:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4674050710 con 0x7f4664009970 2023-10-19T10:37:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46640148b0 con 0x7f46640047c0 2023-10-19T10:37:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f005ffff640 1 -- 172.21.15.50:0/3533734532 shutdown_connections 2023-10-19T10:37:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 -- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050048170 msgr2=0x7f4050038270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 --2- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050048170 0x7f4050038270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f405802c740 tx=0x7f40580677b0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 -- 172.21.15.50:0/2723222343 shutdown_connections 2023-10-19T10:37:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 --2- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4050048170 0x7f4050038270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 --2- 172.21.15.50:0/2723222343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f405004fc40 0x7f4050006ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 -- 172.21.15.50:0/2723222343 >> 172.21.15.50:0/2723222343 conn(0x7f4050054cd0 msgr2=0x7f40500550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 -- 172.21.15.50:0/2723222343 shutdown_connections 2023-10-19T10:37:20.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f47a05f0640 1 -- 172.21.15.50:0/3214757684 shutdown_connections 2023-10-19T10:37:20.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f405d42f640 1 -- 172.21.15.50:0/2723222343 wait complete. 2023-10-19T10:37:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f005ffff640 1 -- 172.21.15.50:0/3533734532 wait complete. 2023-10-19T10:37:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f467affd640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f4664003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f467affd640 1 -- 172.21.15.50:0/2907325834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46640047c0 msgr2=0x7f4664004c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-19T10:37:20.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f467affd640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46640047c0 0x7f4664004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f467affd640 1 -- 172.21.15.50:0/2907325834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664009970 msgr2=0x7f46640051a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-19T10:37:20.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f467affd640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664009970 0x7f46640051a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f467affd640 1 -- 172.21.15.50:0/2907325834 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740509f0 con 0x7f4664003990 2023-10-19T10:37:20.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7f467bfff640 1 -- 172.21.15.50:0/2907325834 reap_dead start 2023-10-19T10:37:20.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7fa1db258640 1 -- 172.21.15.50:0/2952390330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4161800 msgr2=0x7fa1d40c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7fa1db258640 1 --2- 172.21.15.50:0/2952390330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4161800 0x7fa1d40c8660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1d406d950 tx=0x7fa1cc02b7e0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fa1db258640 1 -- 172.21.15.50:0/2952390330 shutdown_connections 2023-10-19T10:37:20.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fa1db258640 1 --2- 172.21.15.50:0/2952390330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4161800 0x7fa1d40c8660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fa1db258640 1 --2- 172.21.15.50:0/2952390330 >> v2:172.21.15.183:3300/0 conn(0x7fa1d415aec0 0x7fa1d40ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fa1db258640 1 -- 172.21.15.50:0/2952390330 >> 172.21.15.50:0/2952390330 conn(0x7fa1d4069e90 msgr2=0x7fa1d40afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.698+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/2122610678 wait complete. 2023-10-19T10:37:20.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.699+0000 7f47a05f0640 1 -- 172.21.15.50:0/3214757684 wait complete. 2023-10-19T10:37:20.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7ff6ffd3e640 1 Processor -- start 2023-10-19T10:37:20.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fa1db258640 1 -- 172.21.15.50:0/2952390330 shutdown_connections 2023-10-19T10:37:20.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7ff545636640 1 Processor -- start 2023-10-19T10:37:20.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7fa1db258640 1 -- 172.21.15.50:0/2952390330 wait complete. 2023-10-19T10:37:20.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7ff545636640 1 -- start start 2023-10-19T10:37:20.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f005ffff640 1 Processor -- start 2023-10-19T10:37:20.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f005ffff640 1 -- start start 2023-10-19T10:37:20.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f47a05f0640 1 Processor -- start 2023-10-19T10:37:20.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f47a05f0640 1 -- start start 2023-10-19T10:37:20.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fd43720f640 1 Processor -- start 2023-10-19T10:37:20.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.700+0000 7fd43720f640 1 -- start start 2023-10-19T10:37:20.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff545636640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400b1830 0x7ff54015a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff545636640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff5400d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7fd43720f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd43006a910 0x7fd4300b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7fd43720f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4300af210 0x7fd4300c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd43720f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4300b4590 0x7fd4300b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd43720f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4300cc7c0 con 0x7fd4300af210 2023-10-19T10:37:20.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd43720f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43015fc60 con 0x7fd4300b4590 2023-10-19T10:37:20.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd43720f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4300b2a40 con 0x7fd43006a910 2023-10-19T10:37:20.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff545636640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5400d1570 0x7ff5400d0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd435a0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4300af210 0x7fd4300c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd435a0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4300af210 0x7fd4300c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38896/0 (socket says 172.21.15.50:38896) 2023-10-19T10:37:20.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd435a0c640 1 -- 172.21.15.50:0/835930362 learned_addr learned my addr 172.21.15.50:0/835930362 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd436a0e640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4300b4590 0x7fd4300b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd43620d640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd43006a910 0x7fd4300b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f467affd640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f4664003dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f466c009bf0 tx=0x7f46640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e94479f17b564 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c02c800 con 0x7f4664003990 2023-10-19T10:37:20.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c02c960 con 0x7f4664003990 2023-10-19T10:37:20.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f4679ffb640 1 -- 172.21.15.50:0/2907325834 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c02ccd0 con 0x7f4664003990 2023-10-19T10:37:20.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 msgr2=0x7f4664003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f4681899640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f4664003dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f466c009bf0 tx=0x7f46640057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff545636640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015d750 con 0x7ff5400cc8c0 2023-10-19T10:37:20.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff545636640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015da30 con 0x7ff5400d1570 2023-10-19T10:37:20.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff545636640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015dd10 con 0x7ff5400b1830 2023-10-19T10:37:20.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd436a0e640 1 -- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd43006a910 msgr2=0x7fd4300b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd436a0e640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd43006a910 0x7fd4300b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7fd436a0e640 1 -- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4300af210 msgr2=0x7fd4300c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7ff6ffd3e640 1 -- start start 2023-10-19T10:37:20.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff53e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff5400d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff53e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff5400d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38906/0 (socket says 172.21.15.50:38906) 2023-10-19T10:37:20.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff53e7fc640 1 -- 172.21.15.50:0/1581715832 learned_addr learned my addr 172.21.15.50:0/1581715832 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7ff53ffff640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5400d1570 0x7ff5400d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd436a0e640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4300af210 0x7fd4300c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd436a0e640 1 -- 172.21.15.50:0/835930362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428009400 con 0x7fd4300b4590 2023-10-19T10:37:20.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd435a0c640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4300af210 0x7fd4300c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:20.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fa1db258640 1 Processor -- start 2023-10-19T10:37:20.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f405d42f640 1 Processor -- start 2023-10-19T10:37:20.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.701+0000 7f405d42f640 1 -- start start 2023-10-19T10:37:20.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f405d42f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4050048170 0x7f40500e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f405d42f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f405004fc40 0x7f405004e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f405d42f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40500e76b0 0x7f40500487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f405d42f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405004d560 con 0x7f40500e76b0 2023-10-19T10:37:20.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f405d42f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405004c950 con 0x7f405004fc40 2023-10-19T10:37:20.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.702+0000 7f405d42f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405004cc30 con 0x7f4050048170 2023-10-19T10:37:20.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4056ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4050048170 0x7f40500e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f40567fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f405004fc40 0x7f405004e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4057fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40500e76b0 0x7f40500487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4057fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40500e76b0 0x7f40500487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38918/0 (socket says 172.21.15.50:38918) 2023-10-19T10:37:20.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4057fff640 1 -- 172.21.15.50:0/2963565337 learned_addr learned my addr 172.21.15.50:0/2963565337 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fa1db258640 1 -- start start 2023-10-19T10:37:20.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 shutdown_connections 2023-10-19T10:37:20.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4681899640 1 --2- 172.21.15.50:0/2907325834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f4664003dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 >> 172.21.15.50:0/2907325834 conn(0x7f4674055de0 msgr2=0x7f46740561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd436a0e640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4300b4590 0x7fd4300b3ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c3c0 tx=0x7fd42000c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff6ffd3e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 0x7ff6f8154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff6ffd3e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6f806d9b0 0x7ff6f81511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f47a05f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f479806ff10 0x7f47981672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff53e7fc640 1 -- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400b1830 msgr2=0x7ff54015a270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-19T10:37:20.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff53e7fc640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5400b1830 0x7ff54015a270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff53e7fc640 1 -- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5400d1570 msgr2=0x7ff5400d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff53e7fc640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5400d1570 0x7ff5400d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff53e7fc640 1 -- 172.21.15.50:0/1581715832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300094d0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 shutdown_connections 2023-10-19T10:37:20.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f47a05f0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f479816cb30 0x7f4798169380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f479e38c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f479806ff10 0x7f47981672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f005ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f00580dc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f40567fc640 1 -- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4050048170 msgr2=0x7f40500e23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f479e38c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f479806ff10 0x7f47981672b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38934/0 (socket says 172.21.15.50:38934) 2023-10-19T10:37:20.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f005ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0058006880 0x7f005804a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f40567fc640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4050048170 0x7f40500e23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f40567fc640 1 -- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40500e76b0 msgr2=0x7f40500487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f40567fc640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40500e76b0 0x7f40500487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f40567fc640 1 -- 172.21.15.50:0/2963565337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4058073490 con 0x7f405004fc40 2023-10-19T10:37:20.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f479e38c640 1 -- 172.21.15.50:0/1989752922 learned_addr learned my addr 172.21.15.50:0/1989752922 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7ff6fd2d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6f806d9b0 0x7ff6f81511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6fdada640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 0x7ff6f8154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6fd2d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6f806d9b0 0x7ff6f81511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38920/0 (socket says 172.21.15.50:38920) 2023-10-19T10:37:20.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6fd2d9640 1 -- 172.21.15.50:0/4151406689 learned_addr learned my addr 172.21.15.50:0/4151406689 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6f80c82c0 0x7ff6f8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f47a05f0640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479816d6d0 0x7f47980c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f40567fc640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f405004fc40 0x7f405004e520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e850 tx=0x7f404c00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f479db8b640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f479816cb30 0x7f4798169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff53e7fc640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff5400d0310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff53400b410 tx=0x7ff53400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8154970 con 0x7ff6f806d9b0 2023-10-19T10:37:20.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8153fa0 con 0x7ff6f806d460 2023-10-19T10:37:20.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8151d40 con 0x7ff6f80c82c0 2023-10-19T10:37:20.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798169bd0 con 0x7f479806ff10 2023-10-19T10:37:20.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798169eb0 con 0x7f479816d6d0 2023-10-19T10:37:20.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534011830 con 0x7ff5400cc8c0 2023-10-19T10:37:20.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f479eb8d640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479816d6d0 0x7f47980c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479816a190 con 0x7f479816cb30 2023-10-19T10:37:20.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5340119b0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff545636640 1 -- 172.21.15.50:0/1581715832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54015dff0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534011d60 con 0x7ff5400cc8c0 2023-10-19T10:37:20.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff545636640 1 -- 172.21.15.50:0/1581715832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54015a7b0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f005effd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f00580dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7f005effd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f00580dc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43214/0 (socket says 172.21.15.50:43214) 2023-10-19T10:37:20.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1db258640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d40cadf0 0x7fa1d40c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1db258640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d415aec0 0x7fa1d4163220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0058006880 0x7f005804a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1db258640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4161800 0x7fa1d4163760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd420005ce0 con 0x7fd4300b4590 2023-10-19T10:37:20.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd43720f640 1 -- 172.21.15.50:0/835930362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4300b2d20 con 0x7fd4300b4590 2023-10-19T10:37:20.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7fd43720f640 1 -- 172.21.15.50:0/835930362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4300b32e0 con 0x7fd4300b4590 2023-10-19T10:37:20.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd420005e40 con 0x7fd4300b4590 2023-10-19T10:37:20.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4200072f0 con 0x7fd4300b4590 2023-10-19T10:37:20.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0095b0 con 0x7f405004fc40 2023-10-19T10:37:20.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1db258640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d41f5d70 con 0x7fa1d40cadf0 2023-10-19T10:37:20.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1db258640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40c38c0 con 0x7fa1d415aec0 2023-10-19T10:37:20.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1db258640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40c3ba0 con 0x7fa1d4161800 2023-10-19T10:37:20.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c00c940 con 0x7f405004fc40 2023-10-19T10:37:20.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.703+0000 7f4681899640 1 -- 172.21.15.50:0/2907325834 wait complete. 2023-10-19T10:37:20.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1d97f5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4161800 0x7fa1d4163760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f4681899640 1 Processor -- start 2023-10-19T10:37:20.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1d97f5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4161800 0x7fa1d4163760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43250/0 (socket says 172.21.15.50:43250) 2023-10-19T10:37:20.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0058006880 0x7f005804a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38948/0 (socket says 172.21.15.50:38948) 2023-10-19T10:37:20.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1d97f5640 1 -- 172.21.15.50:0/24735879 learned_addr learned my addr 172.21.15.50:0/24735879 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 0x7f0058038c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005ffff640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058048a60 con 0x7f0058006880 2023-10-19T10:37:20.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005ffff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005804ab40 con 0x7f00580eac80 2023-10-19T10:37:20.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005ffff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005804ae20 con 0x7f0058005ec0 2023-10-19T10:37:20.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d415aec0 0x7fa1d4163220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f005effd640 1 -- 172.21.15.50:0/2774369365 learned_addr learned my addr 172.21.15.50:0/2774369365 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00ccd0 con 0x7f405004fc40 2023-10-19T10:37:20.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.704+0000 7ff6fe2db640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6f80c82c0 0x7ff6f8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fe2db640 1 -- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 msgr2=0x7ff6f8154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fe2db640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6f806d460 0x7ff6f8154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fe2db640 1 -- 172.21.15.50:0/4151406689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6f806d9b0 msgr2=0x7ff6f81511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fe2db640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6f806d9b0 0x7ff6f81511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fe2db640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0007860 con 0x7ff6f80c82c0 2023-10-19T10:37:20.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fd2d9640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6f806d9b0 0x7ff6f81511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7ff6fe2db640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6f80c82c0 0x7ff6f8151730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0036000 tx=0x7ff6f0002350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0042630 con 0x7ff6f80c82c0 2023-10-19T10:37:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00427b0 con 0x7ff6f80c82c0 2023-10-19T10:37:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0042b60 con 0x7ff6f80c82c0 2023-10-19T10:37:20.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f8153950 con 0x7ff6f80c82c0 2023-10-19T10:37:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7ff545636640 1 -- 172.21.15.50:0/1581715832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff520000fc0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005effd640 1 -- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 msgr2=0x7f0058038c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005effd640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00580eac80 0x7f0058038c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005effd640 1 -- 172.21.15.50:0/2774369365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0058006880 msgr2=0x7f005804a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005effd640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0058006880 0x7f005804a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005effd640 1 -- 172.21.15.50:0/2774369365 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054009400 con 0x7f0058005ec0 2023-10-19T10:37:20.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005e7fc640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0058006880 0x7f005804a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:20.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f8152020 con 0x7ff6f80c82c0 2023-10-19T10:37:20.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7fd43720f640 1 -- 172.21.15.50:0/835930362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4300adf20 con 0x7fd4300b4590 2023-10-19T10:37:20.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f4681899640 1 -- start start 2023-10-19T10:37:20.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f005effd640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f00580dc320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f005002bf50 tx=0x7f0050004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7fa1d8ff4640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d40cadf0 0x7fa1d40c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7fa1d3fff640 1 -- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4161800 msgr2=0x7fa1d4163760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f479e38c640 1 -- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f479816cb30 msgr2=0x7f4798169380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f479e38c640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f479816cb30 0x7f4798169380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f479e38c640 1 -- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479816d6d0 msgr2=0x7f47980c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f479e38c640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f479816d6d0 0x7f47980c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.705+0000 7f479e38c640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479400a590 con 0x7f479806ff10 2023-10-19T10:37:20.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.706+0000 7f479e38c640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f479806ff10 0x7f47981672b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f479400dd90 tx=0x7f479400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7fa1d3fff640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4161800 0x7fa1d4163760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7fa1d3fff640 1 -- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d40cadf0 msgr2=0x7fa1d40c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794012070 con 0x7f479806ff10 2023-10-19T10:37:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479400de80 con 0x7f479806ff10 2023-10-19T10:37:20.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479400b7c0 con 0x7f479806ff10 2023-10-19T10:37:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7fa1d3fff640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d40cadf0 0x7fa1d40c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7fa1d3fff640 1 -- 172.21.15.50:0/24735879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0094d0 con 0x7fa1d415aec0 2023-10-19T10:37:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f4681899640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f46741671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff53401a400 con 0x7ff5400cc8c0 2023-10-19T10:37:20.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405004cf10 con 0x7f405004fc40 2023-10-19T10:37:20.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405004aff0 con 0x7f405004fc40 2023-10-19T10:37:20.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6e0000fc0 con 0x7ff6f80c82c0 2023-10-19T10:37:20.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4788009400 con 0x7f479806ff10 2023-10-19T10:37:20.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479816a750 con 0x7f479806ff10 2023-10-19T10:37:20.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd420007450 con 0x7fd4300b4590 2023-10-19T10:37:20.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f467affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f46741671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7f467affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f46741671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38968/0 (socket says 172.21.15.50:38968) 2023-10-19T10:37:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f467affd640 1 -- 172.21.15.50:0/2130049862 learned_addr learned my addr 172.21.15.50:0/2130049862 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:20.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005003e040 con 0x7f0058005ec0 2023-10-19T10:37:20.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f4681899640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4674167720 0x7f4674053f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f4681899640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674054490 0x7f4674173fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4674054940 con 0x7f4664003990 2023-10-19T10:37:20.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467404de80 con 0x7f4674054490 2023-10-19T10:37:20.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467404d270 con 0x7f4674167720 2023-10-19T10:37:20.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0050009400 con 0x7f0058005ec0 2023-10-19T10:37:20.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fa1d97f5640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4161800 0x7fa1d4163760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:20.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005804b3e0 con 0x7f0058005ec0 2023-10-19T10:37:20.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fd426ffd640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd41406a050 0x7fd41406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd420087f80 con 0x7fd4300b4590 2023-10-19T10:37:20.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fa1d3fff640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d415aec0 0x7fa1d4163220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c500 tx=0x7fa1bc00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050039c30 con 0x7f0058005ec0 2023-10-19T10:37:20.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fd426ffd640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7fd414073d30 0x7fd414076170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.707+0000 7ff5277fe640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff52c06a150 0x7ff52c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff5340888b0 con 0x7ff5400cc8c0 2023-10-19T10:37:20.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff5277fe640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7ff52c073db0 0x7ff52c0761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 --> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff52c0768c0 con 0x7ff52c073db0 2023-10-19T10:37:20.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff534009200 con 0x7ff5400cc8c0 2023-10-19T10:37:20.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff53ffff640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7ff52c073db0 0x7ff52c0761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff53effd640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff52c06a150 0x7ff52c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7f467bfff640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674054490 0x7f4674173fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 --> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd414076840 con 0x7fd414073d30 2023-10-19T10:37:20.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd420088330 con 0x7fd4300b4590 2023-10-19T10:37:20.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00500443f0 con 0x7f0058005ec0 2023-10-19T10:37:20.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f467affd640 1 -- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4674167720 msgr2=0x7f4674053f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-19T10:37:20.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f467affd640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4674167720 0x7f4674053f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f467affd640 1 -- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674054490 msgr2=0x7f4674173fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f467affd640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674054490 0x7f4674173fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f467affd640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467404d550 con 0x7f4664003990 2023-10-19T10:37:20.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff53ffff640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7ff52c073db0 0x7ff52c0761f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7fd43620d640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd41406a050 0x7fd41406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.708+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff6f0042cc0 con 0x7ff6f80c82c0 2023-10-19T10:37:20.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff6ee7fc640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff6dc06a080 0x7ff6dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff53effd640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff52c06a150 0x7ff52c06c610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff5380093a0 tx=0x7ff538009330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f467affd640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f46741671e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f466c02c3b0 tx=0x7f466c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff6f00ab400 con 0x7ff6f80c82c0 2023-10-19T10:37:20.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0035f0 con 0x7fa1d415aec0 2023-10-19T10:37:20.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff6ee7fc640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7ff6dc073d60 0x7ff6dc0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6dc076870 con 0x7ff6dc073d60 2023-10-19T10:37:20.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f00ada20 con 0x7ff6f80c82c0 2023-10-19T10:37:20.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0040000fc0 con 0x7f0058005ec0 2023-10-19T10:37:20.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc003750 con 0x7fa1d415aec0 2023-10-19T10:37:20.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0035f0 con 0x7fa1d415aec0 2023-10-19T10:37:20.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d40c3e80 con 0x7fa1d415aec0 2023-10-19T10:37:20.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7fd43620d640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd41406a050 0x7fd41406c510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd428036000 tx=0x7fd42803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d40c4440 con 0x7fa1d415aec0 2023-10-19T10:37:20.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40500e7a90 con 0x7f405004fc40 2023-10-19T10:37:20.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== osd.6 v2:172.21.15.183:6816/265753870 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff52c0768c0 con 0x7ff52c073db0 2023-10-19T10:37:20.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7f478cff9640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47980cc800 con 0x7f479806ff10 2023-10-19T10:37:20.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c02c800 con 0x7f4664003990 2023-10-19T10:37:20.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c035c30 con 0x7f4664003990 2023-10-19T10:37:20.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c03e410 con 0x7f4664003990 2023-10-19T10:37:20.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.709+0000 7ff6fdada640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff6dc06a080 0x7ff6dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7ff6fd2d9640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7ff6dc073d60 0x7ff6dc0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.710+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f005004d3b0 con 0x7f0058005ec0 2023-10-19T10:37:20.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7fd435a0c640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7fd414073d30 0x7fd414076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1d4164bd0 con 0x7fa1d415aec0 2023-10-19T10:37:20.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f00477fe640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f004c06a120 0x7f004c06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7ff6fd2d9640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7ff6dc073d60 0x7ff6dc0761a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7ff6fdada640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff6dc06a080 0x7ff6dc06c540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6f81e71a0 tx=0x7ff6e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f479400b920 con 0x7f479806ff10 2023-10-19T10:37:20.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467404d830 con 0x7f4664003990 2023-10-19T10:37:20.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f00500aae20 con 0x7f0058005ec0 2023-10-19T10:37:20.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467404ceb0 con 0x7f4664003990 2023-10-19T10:37:20.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f005e7fc640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f004c06a120 0x7f004c06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f00477fe640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f004c073e00 0x7f004c076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 --> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f004c076910 con 0x7f004c073e00 2023-10-19T10:37:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00500ad3a0 con 0x7f0058005ec0 2023-10-19T10:37:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.711+0000 7fd435a0c640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7fd414073d30 0x7fd414076170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f478effd640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f477c06e4b0 0x7f477c070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f005f7fe640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f004c073e00 0x7f004c076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4794089190 con 0x7f479806ff10 2023-10-19T10:37:20.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f478effd640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f477c078110 0x7f477c07a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f477c07ac20 con 0x7f477c078110 2023-10-19T10:37:20.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f479db8b640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f477c06e4b0 0x7f477c070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479400cea0 con 0x7f479806ff10 2023-10-19T10:37:20.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f005e7fc640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f004c06a120 0x7f004c06c5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0058180010 tx=0x7f0054009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f479eb8d640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f477c078110 0x7f477c07a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7f005f7fe640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f004c073e00 0x7f004c076240 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.712+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa1bc036b30 con 0x7fa1d415aec0 2023-10-19T10:37:20.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== osd.0 v2:172.21.15.50:6813/231212571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd414076840 con 0x7fd414073d30 2023-10-19T10:37:20.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4674050cd0 con 0x7f4664003990 2023-10-19T10:37:20.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7f479db8b640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f477c06e4b0 0x7f477c070970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4790001560 tx=0x7f4790009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== osd.4 v2:172.21.15.183:6800/2392619067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff6dc076870 con 0x7ff6dc073d60 2023-10-19T10:37:20.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7f479eb8d640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f477c078110 0x7f477c07a550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7fa1d17fa640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa1b406a110 0x7fa1b406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa1bc087c10 con 0x7fa1d415aec0 2023-10-19T10:37:20.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7fa1d17fa640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fa1b4073cf0 0x7fa1b4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 --> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1b4076800 con 0x7fa1b4073cf0 2023-10-19T10:37:20.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.713+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1bc0035f0 con 0x7fa1d415aec0 2023-10-19T10:37:20.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7fa1d97f5640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fa1b4073cf0 0x7fa1b4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== osd.7 v2:172.21.15.183:6824/3403463202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f004c076910 con 0x7f004c073e00 2023-10-19T10:37:20.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f404c005990 con 0x7f405004fc40 2023-10-19T10:37:20.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7fa1d8ff4640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa1b406a110 0x7fa1b406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== osd.5 v2:172.21.15.183:6808/3486412186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f477c07ac20 con 0x7f477c078110 2023-10-19T10:37:20.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7f403f7fe640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f404406a120 0x7f404406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f404c088e00 con 0x7f405004fc40 2023-10-19T10:37:20.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f403f7fe640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f4044073c90 0x7f40440760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7fa1d97f5640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fa1b4073cf0 0x7fa1b4076130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== osd.3 v2:172.21.15.50:6800/945885036 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa1b4076800 con 0x7fa1b4073cf0 2023-10-19T10:37:20.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f4057fff640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f4044073c90 0x7f40440760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f466c03b050 con 0x7f4664003990 2023-10-19T10:37:20.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.714+0000 7f4663fff640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f466806a080 0x7f466806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f466c0aa4f0 con 0x7f4664003990 2023-10-19T10:37:20.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f4663fff640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] conn(0x7f4668073d60 0x7f46680761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:20.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4668076870 con 0x7f4668073d60 2023-10-19T10:37:20.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f466c0aca50 con 0x7f4664003990 2023-10-19T10:37:20.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.715+0000 7f467bfff640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] conn(0x7f4668073d60 0x7f46680761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.716+0000 7f467bfff640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] conn(0x7f4668073d60 0x7f46680761a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.716+0000 7f4057fff640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f4044073c90 0x7f40440760d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.716+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 --> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40440767a0 con 0x7f4044073c90 2023-10-19T10:37:20.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.717+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404c0891b0 con 0x7f405004fc40 2023-10-19T10:37:20.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.717+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== osd.2 v2:172.21.15.50:6804/415608244 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4668076870 con 0x7f4668073d60 2023-10-19T10:37:20.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.717+0000 7f4056ffd640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f404406a120 0x7f404406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.717+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== osd.1 v2:172.21.15.50:6824/3823862985 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f40440767a0 con 0x7f4044073c90 2023-10-19T10:37:20.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.717+0000 7fa1d8ff4640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa1b406a110 0x7fa1b406c5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40075b0 tx=0x7fa1c4007540 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.717+0000 7f467a7fc640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f466806a080 0x7f466806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:20.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.721+0000 7f4056ffd640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f404406a120 0x7f404406c5e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4048000f60 tx=0x7f4048007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.721+0000 7f467a7fc640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f466806a080 0x7f466806c540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4674168230 tx=0x7f4670001470 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:20.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.741+0000 7fd43720f640 1 -- 172.21.15.50:0/835930362 --> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd4300adc40 con 0x7fd414073d30 2023-10-19T10:37:20.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.741+0000 7fd426ffd640 1 -- 172.21.15.50:0/835930362 <== osd.0 v2:172.21.15.50:6813/231212571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd42c00d400 con 0x7fd414073d30 2023-10-19T10:37:20.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7fd414073d30 msgr2=0x7fd414076170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7fd414073d30 0x7fd414076170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd41406a050 msgr2=0x7fd41406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd41406a050 0x7fd41406c510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd428036000 tx=0x7fd42803c040 comp rx=0 tx=0).stop 2023-10-19T10:37:20.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4300b4590 msgr2=0x7fd4300b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 --2- 172.21.15.50:0/835930362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4300b4590 0x7fd4300b3ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c3c0 tx=0x7fd42000c7c0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd436a0e640 1 -- 172.21.15.50:0/835930362 reap_dead start 2023-10-19T10:37:20.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 shutdown_connections 2023-10-19T10:37:20.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.742+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 >> 172.21.15.50:0/835930362 conn(0x7fd4300cff20 msgr2=0x7fd4300af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.743+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 shutdown_connections 2023-10-19T10:37:20.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.743+0000 7fd424ff9640 1 -- 172.21.15.50:0/835930362 wait complete. 2023-10-19T10:37:20.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.743+0000 7ff545636640 1 -- 172.21.15.50:0/1581715832 --> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff520002cb0 con 0x7ff52c073db0 2023-10-19T10:37:20.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.744+0000 7ff5277fe640 1 -- 172.21.15.50:0/1581715832 <== osd.6 v2:172.21.15.183:6816/265753870 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff520002cb0 con 0x7ff52c073db0 2023-10-19T10:37:20.866 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.745+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 --> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6e0002cb0 con 0x7ff6dc073d60 2023-10-19T10:37:20.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.745+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7ff52c073db0 msgr2=0x7ff52c0761f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.745+0000 7ff5257fa640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7ff52c073db0 0x7ff52c0761f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.745+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff52c06a150 msgr2=0x7ff52c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.745+0000 7ff5257fa640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff52c06a150 0x7ff52c06c610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff5380093a0 tx=0x7ff538009330 comp rx=0 tx=0).stop 2023-10-19T10:37:20.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 msgr2=0x7ff5400d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff5257fa640 1 --2- 172.21.15.50:0/1581715832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5400cc8c0 0x7ff5400d0310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff53400b410 tx=0x7ff53400b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff53ffff640 1 -- 172.21.15.50:0/1581715832 reap_dead start 2023-10-19T10:37:20.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 shutdown_connections 2023-10-19T10:37:20.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 >> 172.21.15.50:0/1581715832 conn(0x7ff540070f30 msgr2=0x7ff5400b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 shutdown_connections 2023-10-19T10:37:20.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff5257fa640 1 -- 172.21.15.50:0/1581715832 wait complete. 2023-10-19T10:37:20.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7ff6ee7fc640 1 -- 172.21.15.50:0/4151406689 <== osd.4 v2:172.21.15.183:6800/2392619067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff6e0002cb0 con 0x7ff6dc073d60 2023-10-19T10:37:20.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.746+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 --> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa1d41648f0 con 0x7fa1b4073cf0 2023-10-19T10:37:20.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7ff6dc073d60 msgr2=0x7ff6dc0761a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7ff6dc073d60 0x7ff6dc0761a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7fa1d17fa640 1 -- 172.21.15.50:0/24735879 <== osd.3 v2:172.21.15.50:6800/945885036 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa1cc004500 con 0x7fa1b4073cf0 2023-10-19T10:37:20.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff6dc06a080 msgr2=0x7ff6dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff6dc06a080 0x7ff6dc06c540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6f81e71a0 tx=0x7ff6e8008040 comp rx=0 tx=0).stop 2023-10-19T10:37:20.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6f80c82c0 msgr2=0x7ff6f8151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6ffd3e640 1 --2- 172.21.15.50:0/4151406689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6f80c82c0 0x7ff6f8151730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0036000 tx=0x7ff6f0002350 comp rx=0 tx=0).stop 2023-10-19T10:37:20.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.747+0000 7ff6fe2db640 1 -- 172.21.15.50:0/4151406689 reap_dead start 2023-10-19T10:37:20.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 shutdown_connections 2023-10-19T10:37:20.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 >> 172.21.15.50:0/4151406689 conn(0x7ff6f80cdc70 msgr2=0x7ff6f8154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fa1b4073cf0 msgr2=0x7fa1b4076130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1db258640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fa1b4073cf0 0x7fa1b4076130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 shutdown_connections 2023-10-19T10:37:20.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7ff6ffd3e640 1 -- 172.21.15.50:0/4151406689 wait complete. 2023-10-19T10:37:20.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 --> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0040002cb0 con 0x7f004c073e00 2023-10-19T10:37:20.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7f478cff9640 1 -- 172.21.15.50:0/1989752922 --> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f479806fc30 con 0x7f477c078110 2023-10-19T10:37:20.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa1b406a110 msgr2=0x7fa1b406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1db258640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa1b406a110 0x7fa1b406c5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40075b0 tx=0x7fa1c4007540 comp rx=0 tx=0).stop 2023-10-19T10:37:20.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d415aec0 msgr2=0x7fa1d4163220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1db258640 1 --2- 172.21.15.50:0/24735879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d415aec0 0x7fa1d4163220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c500 tx=0x7fa1bc00c9c0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.748+0000 7fa1d97f5640 1 -- 172.21.15.50:0/24735879 reap_dead start 2023-10-19T10:37:20.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.749+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 shutdown_connections 2023-10-19T10:37:20.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.749+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 >> 172.21.15.50:0/24735879 conn(0x7fa1d4069e90 msgr2=0x7fa1d40afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.749+0000 7f00477fe640 1 -- 172.21.15.50:0/2774369365 <== osd.7 v2:172.21.15.183:6824/3403463202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0040002cb0 con 0x7f004c073e00 2023-10-19T10:37:20.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.749+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 shutdown_connections 2023-10-19T10:37:20.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.749+0000 7fa1db258640 1 -- 172.21.15.50:0/24735879 wait complete. 2023-10-19T10:37:20.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.749+0000 7f478effd640 1 -- 172.21.15.50:0/1989752922 <== osd.5 v2:172.21.15.183:6808/3486412186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4788004040 con 0x7f477c078110 2023-10-19T10:37:20.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f004c073e00 msgr2=0x7f004c076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f004c073e00 0x7f004c076240 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 --> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f467404dfc0 con 0x7f4668073d60 2023-10-19T10:37:20.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f477c078110 msgr2=0x7f477c07a550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f47a05f0640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f477c078110 0x7f477c07a550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f004c06a120 msgr2=0x7f004c06c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f004c06a120 0x7f004c06c5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0058180010 tx=0x7f0054009190 comp rx=0 tx=0).stop 2023-10-19T10:37:20.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 msgr2=0x7f00580dc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 --2- 172.21.15.50:0/2774369365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0058005ec0 0x7f00580dc320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f005002bf50 tx=0x7f0050004730 comp rx=0 tx=0).stop 2023-10-19T10:37:20.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005f7fe640 1 -- 172.21.15.50:0/2774369365 reap_dead start 2023-10-19T10:37:20.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 shutdown_connections 2023-10-19T10:37:20.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 >> 172.21.15.50:0/2774369365 conn(0x7f0058009fe0 msgr2=0x7f0058039c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f477c06e4b0 msgr2=0x7f477c070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f47a05f0640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f477c06e4b0 0x7f477c070970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4790001560 tx=0x7f4790009040 comp rx=0 tx=0).stop 2023-10-19T10:37:20.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f4663fff640 1 -- 172.21.15.50:0/2130049862 <== osd.2 v2:172.21.15.50:6804/415608244 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f467c04eac0 con 0x7f4668073d60 2023-10-19T10:37:20.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f479806ff10 msgr2=0x7f47981672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f47a05f0640 1 --2- 172.21.15.50:0/1989752922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f479806ff10 0x7f47981672b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f479400dd90 tx=0x7f479400c7f0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.750+0000 7f479eb8d640 1 -- 172.21.15.50:0/1989752922 reap_dead start 2023-10-19T10:37:20.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 shutdown_connections 2023-10-19T10:37:20.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 >> 172.21.15.50:0/1989752922 conn(0x7f47980cdfd0 msgr2=0x7f479806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 shutdown_connections 2023-10-19T10:37:20.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f47a05f0640 1 -- 172.21.15.50:0/1989752922 wait complete. 2023-10-19T10:37:20.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 shutdown_connections 2023-10-19T10:37:20.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] conn(0x7f4668073d60 msgr2=0x7f46680761a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f4681899640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] conn(0x7f4668073d60 0x7f46680761a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.751+0000 7f005ffff640 1 -- 172.21.15.50:0/2774369365 wait complete. 2023-10-19T10:37:20.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.752+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f466806a080 msgr2=0x7f466806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.752+0000 7f4681899640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f466806a080 0x7f466806c540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4674168230 tx=0x7f4670001470 comp rx=0 tx=0).stop 2023-10-19T10:37:20.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.752+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 msgr2=0x7f46741671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.752+0000 7f4681899640 1 --2- 172.21.15.50:0/2130049862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664003990 0x7f46741671e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f466c02c3b0 tx=0x7f466c004730 comp rx=0 tx=0).stop 2023-10-19T10:37:20.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.752+0000 7f467bfff640 1 -- 172.21.15.50:0/2130049862 reap_dead start 2023-10-19T10:37:20.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.752+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 --> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4050007650 con 0x7f4044073c90 2023-10-19T10:37:20.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.753+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 shutdown_connections 2023-10-19T10:37:20.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.753+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 >> 172.21.15.50:0/2130049862 conn(0x7f4674055de0 msgr2=0x7f46740dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.753+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 shutdown_connections 2023-10-19T10:37:20.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.753+0000 7f4681899640 1 -- 172.21.15.50:0/2130049862 wait complete. 2023-10-19T10:37:20.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.753+0000 7f403f7fe640 1 -- 172.21.15.50:0/2963565337 <== osd.1 v2:172.21.15.50:6824/3823862985 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40580a5bd0 con 0x7f4044073c90 2023-10-19T10:37:20.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f4044073c90 msgr2=0x7f40440760d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f4044073c90 0x7f40440760d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:20.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f404406a120 msgr2=0x7f404406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f404406a120 0x7f404406c5e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4048000f60 tx=0x7f4048007e40 comp rx=0 tx=0).stop 2023-10-19T10:37:20.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f405004fc40 msgr2=0x7f405004e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:20.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 --2- 172.21.15.50:0/2963565337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f405004fc40 0x7f405004e520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e850 tx=0x7f404c00ed10 comp rx=0 tx=0).stop 2023-10-19T10:37:20.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f4057fff640 1 -- 172.21.15.50:0/2963565337 reap_dead start 2023-10-19T10:37:20.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 shutdown_connections 2023-10-19T10:37:20.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.754+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 >> 172.21.15.50:0/2963565337 conn(0x7f4050054cd0 msgr2=0x7f4050053960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:20.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.755+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 shutdown_connections 2023-10-19T10:37:20.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:20.755+0000 7f405d42f640 1 -- 172.21.15.50:0/2963565337 wait complete. 2023-10-19T10:37:20.894 INFO:teuthology.orchestra.run.smithi050.stdout:55834574859 2023-10-19T10:37:20.894 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-19T10:37:20.895 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:20.896 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-19T10:37:20.897 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:20.897 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-19T10:37:20.898 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:20.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-19T10:37:20.899 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2023-10-19T10:37:20.900 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-19T10:37:20.901 INFO:teuthology.orchestra.run.smithi050.stdout:60129542155 2023-10-19T10:37:20.901 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-19T10:37:20.902 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2023-10-19T10:37:20.902 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-19T10:37:20.903 INFO:teuthology.orchestra.run.smithi050.stdout:55834574859 2023-10-19T10:37:20.904 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-19T10:37:21.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.114+0000 7f12f9edc640 1 Processor -- start 2023-10-19T10:37:21.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.115+0000 7f12f9edc640 1 -- start start 2023-10-19T10:37:21.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f9edc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec050380 0x7f12ec007580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f9edc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec0064e0 con 0x7f12ec005fe0 2023-10-19T10:37:21.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f9edc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec0067c0 con 0x7f12ec055730 2023-10-19T10:37:21.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f9edc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec006aa0 con 0x7f12ec050380 2023-10-19T10:37:21.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f96db640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f12ec005fe0 0x7f12ec0381d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33552/0 (socket says 172.21.15.50:33552) 2023-10-19T10:37:21.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f96db640 1 -- 172.21.15.50:0/2989410032 learned_addr learned my addr 172.21.15.50:0/2989410032 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:21.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f12f3fff640 1 --2- 172.21.15.50:0/2989410032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec050380 0x7f12ec007580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7f12f3fff640 1 -- 172.21.15.50:0/2989410032 >> v1:172.21.15.183:6789/0 conn(0x7f12ec055730 legacy=0x7f12ec04fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7f12f3fff640 1 -- 172.21.15.50:0/2989410032 >> v1:172.21.15.50:6789/0 conn(0x7f12ec005fe0 legacy=0x7f12ec0381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7f12f3fff640 1 -- 172.21.15.50:0/2989410032 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec006d80 con 0x7f12ec050380 2023-10-19T10:37:21.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.116+0000 7f1cb4817640 1 Processor -- start 2023-10-19T10:37:21.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7f1cb4817640 1 -- start start 2023-10-19T10:37:21.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7f1cb4817640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca8006b00 0x7f1ca8048b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7fd456815640 1 Processor -- start 2023-10-19T10:37:21.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7fd456815640 1 -- start start 2023-10-19T10:37:21.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7f1cb4817640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1ca80538d0 0x7f1ca8053d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7ff82aee9640 1 Processor -- start 2023-10-19T10:37:21.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.117+0000 7ff82aee9640 1 -- start start 2023-10-19T10:37:21.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7f1cadd9b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f1ca8050460 0x7f1ca8006ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41874/0 (socket says 172.21.15.50:41874) 2023-10-19T10:37:21.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7f1cadd9b640 1 -- 172.21.15.50:0/1638052481 learned_addr learned my addr 172.21.15.50:0/1638052481 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:21.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca80509d0 con 0x7f1ca8050460 2023-10-19T10:37:21.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.118+0000 7f1caed9d640 1 --2- 172.21.15.50:0/1638052481 >> v2:172.21.15.183:3300/0 conn(0x7f1ca80538d0 0x7f1ca8053d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f816f59e640 1 Processor -- start 2023-10-19T10:37:21.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7ff82aee9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 0x7ff81c053910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca8051650 con 0x7f1ca80538d0 2023-10-19T10:37:21.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8050cb0 con 0x7f1ca8006b00 2023-10-19T10:37:21.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670228045 0 0) 0x7f1ca80509d0 con 0x7f1ca8050460 2023-10-19T10:37:21.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f816f59e640 1 -- start start 2023-10-19T10:37:21.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f1cae59c640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca8006b00 0x7f1ca8048b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f12f3fff640 1 --2- 172.21.15.50:0/2989410032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec050380 0x7f12ec007580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12f40788b0 tx=0x7f12f40a50d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bac60a9954c01b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f051b59e640 1 Processor -- start 2023-10-19T10:37:21.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7ff823fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7ff81c058a00 0x7ff81c0577f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41890/0 (socket says 172.21.15.50:41890) 2023-10-19T10:37:21.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7ff823fff640 1 -- 172.21.15.50:0/3803872016 learned_addr learned my addr 172.21.15.50:0/3803872016 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:21.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c98003520 con 0x7f1ca8050460 2023-10-19T10:37:21.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c006c70 con 0x7ff81c007100 2023-10-19T10:37:21.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7f051b59e640 1 -- start start 2023-10-19T10:37:21.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.119+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c053e50 con 0x7ff81c058a00 2023-10-19T10:37:21.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0e7910 con 0x7ff81c053530 2023-10-19T10:37:21.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217777023 0 0) 0x7ff81c006c70 con 0x7ff81c007100 2023-10-19T10:37:21.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f12f37fe640 1 -- 172.21.15.50:0/2989410032 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40a5d30 con 0x7f12ec050380 2023-10-19T10:37:21.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff814003520 con 0x7ff81c007100 2023-10-19T10:37:21.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957160235 0 0) 0x7ff81c053e50 con 0x7ff81c058a00 2023-10-19T10:37:21.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff81c006c70 con 0x7ff81c058a00 2023-10-19T10:37:21.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172775429 0 0) 0x7f1c98003520 con 0x7f1ca8050460 2023-10-19T10:37:21.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca8006b00 msgr2=0x7f1ca8048b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca8006b00 0x7f1ca8048b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 >> v2:172.21.15.183:3300/0 conn(0x7f1ca80538d0 msgr2=0x7f1ca8053d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 --2- 172.21.15.50:0/1638052481 >> v2:172.21.15.183:3300/0 conn(0x7f1ca80538d0 0x7f1ca8053d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8050f90 con 0x7f1ca8050460 2023-10-19T10:37:21.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f1ca4001670 con 0x7f1ca8050460 2023-10-19T10:37:21.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 >> v1:172.21.15.183:6789/0 conn(0x7f1ca8050460 legacy=0x7f1ca8006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98003970 0x7f1c98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c980047c0 0x7f1c98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f12f37fe640 1 -- 172.21.15.50:0/2989410032 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f40a5e90 con 0x7f12ec050380 2023-10-19T10:37:21.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c98005180 0x7f1c9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8051650 con 0x7f1c98003970 2023-10-19T10:37:21.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8050cb0 con 0x7f1c980047c0 2023-10-19T10:37:21.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9800c080 con 0x7f1c98005180 2023-10-19T10:37:21.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c980047c0 0x7f1c98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f12f37fe640 1 -- 172.21.15.50:0/2989410032 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40ad680 con 0x7f12ec050380 2023-10-19T10:37:21.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f816f59e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f817015e4a0 0x7f81700cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 495565598 0 0) 0x7ff814003520 con 0x7ff81c007100 2023-10-19T10:37:21.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cae59c640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98003970 0x7f1c98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f816e59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f817015e4a0 0x7f81700cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f816e59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f817015e4a0 0x7f81700cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:55922/0 (socket says 172.21.15.50:55922) 2023-10-19T10:37:21.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f816e59c640 1 -- 172.21.15.50:0/2931137183 learned_addr learned my addr 172.21.15.50:0/2931137183 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 msgr2=0x7ff81c053910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-19T10:37:21.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 0x7ff81c053910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 >> v1:172.21.15.183:6789/0 conn(0x7ff81c058a00 legacy=0x7ff81c0577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c006060 con 0x7ff81c007100 2023-10-19T10:37:21.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd456815640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd450069e50 0x7fd450160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7ff818001670 con 0x7ff81c007100 2023-10-19T10:37:21.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f051b59e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f051c06af40 0x7f051c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81700cb500 con 0x7f81701614e0 2023-10-19T10:37:21.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817006c5e0 con 0x7f817006b7b0 2023-10-19T10:37:21.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817015e880 con 0x7f817015e4a0 2023-10-19T10:37:21.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fe0ed0bd640 1 Processor -- start 2023-10-19T10:37:21.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 >> v1:172.21.15.50:6789/0 conn(0x7ff81c007100 legacy=0x7ff81c03bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 0x7ff814004bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fe0ed0bd640 1 -- start start 2023-10-19T10:37:21.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff814005170 0x7ff814003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814003dc0 0x7ff81400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd456815640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd4500ce830 0x7fd4500c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f051a59c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f051c06af40 0x7f051c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f051a59c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f051c06af40 0x7f051c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38986/0 (socket says 172.21.15.50:38986) 2023-10-19T10:37:21.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f051a59c640 1 -- 172.21.15.50:0/58331280 learned_addr learned my addr 172.21.15.50:0/58331280 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd456815640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4501643a0 con 0x7fd450069a70 2023-10-19T10:37:21.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd456815640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500cd1d0 con 0x7fd450069e50 2023-10-19T10:37:21.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd456815640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd450161340 con 0x7fd4500ce830 2023-10-19T10:37:21.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810001160 con 0x7ff814005170 2023-10-19T10:37:21.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0e7910 con 0x7ff814003dc0 2023-10-19T10:37:21.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff829486640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814003dc0 0x7ff81400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff814004580 con 0x7ff81c053530 2023-10-19T10:37:21.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cadd9b640 1 -- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c98005180 msgr2=0x7f1c9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c98005180 0x7f1c9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cadd9b640 1 -- 172.21.15.50:0/1638052481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98003970 msgr2=0x7f1c98003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98003970 0x7f1c98003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.120+0000 7f1cadd9b640 1 -- 172.21.15.50:0/1638052481 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8050f90 con 0x7f1c980047c0 2023-10-19T10:37:21.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f051b59e640 1 --2- 172.21.15.50:0/58331280 >> v2:172.21.15.183:3300/0 conn(0x7f051c06e440 0x7f051c069800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7ff828c85640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 0x7ff814004bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090687794 0 0) 0x7f81700cb500 con 0x7f81701614e0 2023-10-19T10:37:21.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f12f9edc640 1 -- 172.21.15.50:0/2989410032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec050380 msgr2=0x7f12ec007580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f12f9edc640 1 --2- 172.21.15.50:0/2989410032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec050380 0x7f12ec007580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12f40788b0 tx=0x7f12f40a50d0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815c003520 con 0x7f81701614e0 2023-10-19T10:37:21.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708063902 0 0) 0x7f817006c5e0 con 0x7f817006b7b0 2023-10-19T10:37:21.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81700cb500 con 0x7f817006b7b0 2023-10-19T10:37:21.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd456014640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd4500ce830 0x7fd4500c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd455813640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd450069a70 0x7fd4500ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33566/0 (socket says 172.21.15.50:33566) 2023-10-19T10:37:21.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.121+0000 7fd455813640 1 -- 172.21.15.50:0/597884596 learned_addr learned my addr 172.21.15.50:0/597884596 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:21.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f12f9edc640 1 -- 172.21.15.50:0/2989410032 shutdown_connections 2023-10-19T10:37:21.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f12f9edc640 1 --2- 172.21.15.50:0/2989410032 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec050380 0x7f12ec007580 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f12f9edc640 1 -- 172.21.15.50:0/2989410032 >> 172.21.15.50:0/2989410032 conn(0x7f12ec054c10 msgr2=0x7f12ec055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880039523 0 0) 0x7f81700cb500 con 0x7f817006b7b0 2023-10-19T10:37:21.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 >> v2:172.21.15.183:3300/0 conn(0x7f817015e4a0 msgr2=0x7f81700cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 --2- 172.21.15.50:0/2931137183 >> v2:172.21.15.183:3300/0 conn(0x7f817015e4a0 0x7f81700cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 >> v1:172.21.15.50:6789/0 conn(0x7f81701614e0 legacy=0x7f81700cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81701619b0 con 0x7f817006b7b0 2023-10-19T10:37:21.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f8158001670 con 0x7f817006b7b0 2023-10-19T10:37:21.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7ff829486640 1 -- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 msgr2=0x7ff814004bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7ff829486640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff81c053530 0x7ff814004bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7ff829486640 1 -- 172.21.15.50:0/3803872016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff814005170 msgr2=0x7ff814003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7ff829486640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff814005170 0x7ff814003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c06d8d0 con 0x7f051c0ce180 2023-10-19T10:37:21.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7ff829486640 1 -- 172.21.15.50:0/3803872016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814003520 con 0x7ff814003dc0 2023-10-19T10:37:21.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f051ad9d640 1 --2- 172.21.15.50:0/58331280 >> v2:172.21.15.183:3300/0 conn(0x7f051c06e440 0x7f051c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c980047c0 0x7f1c98004c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c210 tx=0x7f1c980057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b49a69273b23e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fd455012640 1 --2- 172.21.15.50:0/597884596 >> v2:172.21.15.50:3300/0 conn(0x7fd450069e50 0x7fd450160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051c06d5f0 con 0x7f051c06af40 2023-10-19T10:37:21.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051c0cd160 con 0x7f051c06e440 2023-10-19T10:37:21.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f12f9edc640 1 -- 172.21.15.50:0/2989410032 shutdown_connections 2023-10-19T10:37:21.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca402cb00 con 0x7f1c980047c0 2023-10-19T10:37:21.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca402cc60 con 0x7f1c980047c0 2023-10-19T10:37:21.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f1cad59a640 1 -- 172.21.15.50:0/1638052481 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4005b10 con 0x7f1c980047c0 2023-10-19T10:37:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f051ad9d640 1 -- 172.21.15.50:0/58331280 >> v2:172.21.15.50:3300/0 conn(0x7f051c06af40 msgr2=0x7f051c0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fd454811640 1 -- 172.21.15.50:0/597884596 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849384901 0 0) 0x7fd4501643a0 con 0x7fd450069a70 2023-10-19T10:37:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fd454811640 1 -- 172.21.15.50:0/597884596 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd43c003520 con 0x7fd450069a70 2023-10-19T10:37:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fd455012640 1 -- 172.21.15.50:0/597884596 >> v2:172.21.15.183:3300/0 conn(0x7fd4500ce830 msgr2=0x7fd4500c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f12f9edc640 1 -- 172.21.15.50:0/2989410032 wait complete. 2023-10-19T10:37:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f051ad9d640 1 --2- 172.21.15.50:0/58331280 >> v2:172.21.15.50:3300/0 conn(0x7f051c06af40 0x7f051c0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fd455012640 1 --2- 172.21.15.50:0/597884596 >> v2:172.21.15.183:3300/0 conn(0x7fd4500ce830 0x7fd4500c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fd455012640 1 -- 172.21.15.50:0/597884596 >> v1:172.21.15.50:6789/0 conn(0x7fd450069a70 legacy=0x7fd4500ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fd455012640 1 -- 172.21.15.50:0/597884596 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450161620 con 0x7fd450069e50 2023-10-19T10:37:21.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fd456014640 1 --2- 172.21.15.50:0/597884596 >> v2:172.21.15.183:3300/0 conn(0x7fd4500ce830 0x7fd4500c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:21.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f9b010fa640 1 Processor -- start 2023-10-19T10:37:21.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f051ad9d640 1 -- 172.21.15.50:0/58331280 >> v1:172.21.15.183:6789/0 conn(0x7f051c0ce180 legacy=0x7f051c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fe0ed0bd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe0e80cb290 0x7fe0e80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fe0ed0bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e80ad770 0x7fe0e80cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fe0ed0bd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe0e815dfe0 0x7fe0e80cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fe0ed0bd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e81600e0 con 0x7fe0e80cb290 2023-10-19T10:37:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fe0ed0bd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e8160330 con 0x7fe0e815dfe0 2023-10-19T10:37:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7fe0ed0bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8160610 con 0x7fe0e80ad770 2023-10-19T10:37:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e759e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe0e815dfe0 0x7fe0e80cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e759e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe0e815dfe0 0x7fe0e80cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:55960/0 (socket says 172.21.15.50:55960) 2023-10-19T10:37:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e759e640 1 -- 172.21.15.50:0/1491844371 learned_addr learned my addr 172.21.15.50:0/1491844371 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/1491844371 >> v2:172.21.15.50:3300/0 conn(0x7fe0e80cb290 0x7fe0e80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f051ad9d640 1 -- 172.21.15.50:0/58331280 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0cd440 con 0x7f051c06e440 2023-10-19T10:37:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c980047c0 msgr2=0x7f1c98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f1cb4817640 1 --2- 172.21.15.50:0/1638052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c980047c0 0x7f1c98004c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c210 tx=0x7f1c980057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f1caed9d640 1 -- 172.21.15.50:0/1638052481 reap_dead start 2023-10-19T10:37:21.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7ff829486640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814003dc0 0x7ff81400b980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff818007ed0 tx=0x7ff8140057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2a281acab6cffe server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 >> v1:172.21.15.183:6789/0 conn(0x7f817006b7b0 legacy=0x7f8170160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f815c004b60 0x7f815c004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c003760 0x7f815c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f815c004120 0x7f815c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680015d0 con 0x7f815c004120 2023-10-19T10:37:21.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817015e880 con 0x7f815c003760 2023-10-19T10:37:21.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c00c020 con 0x7f815c004b60 2023-10-19T10:37:21.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f8167fff640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c003760 0x7f815c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816e59c640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f815c004b60 0x7f815c004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.122+0000 7f816ed9d640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f815c004120 0x7f815c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f816e59c640 1 -- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c003760 msgr2=0x7f815c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f816e59c640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c003760 0x7f815c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f816e59c640 1 -- 172.21.15.50:0/2931137183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f815c004120 msgr2=0x7f815c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f816e59c640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f815c004120 0x7f815c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f816e59c640 1 -- 172.21.15.50:0/2931137183 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81701619b0 con 0x7f815c004b60 2023-10-19T10:37:21.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f816ed9d640 1 -- 172.21.15.50:0/2931137183 reap_dead start 2023-10-19T10:37:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0dffff640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e80ad770 0x7fe0e80cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/1491844371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e80ad770 msgr2=0x7fe0e80cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7f9b010fa640 1 -- start start 2023-10-19T10:37:21.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81802c8a0 con 0x7ff814003dc0 2023-10-19T10:37:21.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81802ca20 con 0x7ff814003dc0 2023-10-19T10:37:21.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7ff8237fe640 1 -- 172.21.15.50:0/3803872016 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81802cdd0 con 0x7ff814003dc0 2023-10-19T10:37:21.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 shutdown_connections 2023-10-19T10:37:21.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f051ad9d640 1 --2- 172.21.15.50:0/58331280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c069800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f051c0b12b0 tx=0x7f051402df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2637975d7eaf039 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 >> 172.21.15.50:0/1638052481 conn(0x7f1ca8055c00 msgr2=0x7f1ca8056010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e80ad770 0x7fe0e80cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/1491844371 >> v2:172.21.15.183:3300/0 conn(0x7fe0e815dfe0 msgr2=0x7fe0e80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/1491844371 >> v2:172.21.15.183:3300/0 conn(0x7fe0e815dfe0 0x7fe0e80cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.123+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/1491844371 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e81608f0 con 0x7fe0e80cb290 2023-10-19T10:37:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7fe0dffff640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e80ad770 0x7fe0e80cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:21.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f051959a640 1 -- 172.21.15.50:0/58331280 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051402ea20 con 0x7f051c06e440 2023-10-19T10:37:21.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f051959a640 1 -- 172.21.15.50:0/58331280 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051402eb80 con 0x7f051c06e440 2023-10-19T10:37:21.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f051959a640 1 -- 172.21.15.50:0/58331280 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400a2e0 con 0x7f051c06e440 2023-10-19T10:37:21.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7fd455012640 1 --2- 172.21.15.50:0/597884596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450160d50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd4500ae900 tx=0x7fd44802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe63694e67a5808 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f12f9edc640 1 Processor -- start 2023-10-19T10:37:21.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f12f9edc640 1 -- start start 2023-10-19T10:37:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 shutdown_connections 2023-10-19T10:37:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.124+0000 7f1cb4817640 1 -- 172.21.15.50:0/1638052481 wait complete. 2023-10-19T10:37:21.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fd454811640 1 -- 172.21.15.50:0/597884596 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44802cbe0 con 0x7fd450069e50 2023-10-19T10:37:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e80cb290 0x7fe0e80ad230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80afb70 tx=0x7fe0d802bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e978ad863dd5a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fd454811640 1 -- 172.21.15.50:0/597884596 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44802cd40 con 0x7fd450069e50 2023-10-19T10:37:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 msgr2=0x7f051c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9b010fa640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9af4053150 0x7f9af4006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 --2- 172.21.15.50:0/58331280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c069800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f051c0b12b0 tx=0x7f051402df00 comp rx=0 tx=0).stop 2023-10-19T10:37:21.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f816e59c640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f815c004b60 0x7f815c004f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8160009b70 tx=0x7f815c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf9f8724a64c7f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9b010fa640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af40543f0 con 0x7f9af40066d0 2023-10-19T10:37:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9b008f9640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f9af40066d0 0x7f9af4006b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33582/0 (socket says 172.21.15.50:33582) 2023-10-19T10:37:21.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fe0e659c640 1 -- 172.21.15.50:0/1491844371 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d802d6e0 con 0x7fe0e80cb290 2023-10-19T10:37:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9b008f9640 1 -- 172.21.15.50:0/128600171 learned_addr learned my addr 172.21.15.50:0/128600171 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9afb7fe640 1 --2- 172.21.15.50:0/128600171 >> v2:172.21.15.183:3300/0 conn(0x7f9af4053150 0x7f9af4006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fe0e659c640 1 -- 172.21.15.50:0/1491844371 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d802d840 con 0x7fe0e80cb290 2023-10-19T10:37:21.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814003dc0 msgr2=0x7ff81400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7ff82aee9640 1 --2- 172.21.15.50:0/3803872016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814003dc0 0x7ff81400b980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff818007ed0 tx=0x7ff8140057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 shutdown_connections 2023-10-19T10:37:21.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 --2- 172.21.15.50:0/58331280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c069800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 --2- 172.21.15.50:0/58331280 >> v2:172.21.15.50:3300/0 conn(0x7f051c06af40 0x7f051c0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 >> 172.21.15.50:0/58331280 conn(0x7f051c0b01d0 msgr2=0x7f051c0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f12f9edc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec005fe0 0x7f12ec04dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7ff829486640 1 -- 172.21.15.50:0/3803872016 reap_dead start 2023-10-19T10:37:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af40343a0 con 0x7f9af4033fc0 2023-10-19T10:37:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af40535d0 con 0x7f9af4053150 2023-10-19T10:37:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 shutdown_connections 2023-10-19T10:37:21.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f051b59e640 1 -- 172.21.15.50:0/58331280 wait complete. 2023-10-19T10:37:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7f12f9edc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12ec050380 0x7f12ec0dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fd454811640 1 -- 172.21.15.50:0/597884596 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448034660 con 0x7fd450069e50 2023-10-19T10:37:21.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765660753 0 0) 0x7f9af40543f0 con 0x7f9af40066d0 2023-10-19T10:37:21.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aec003520 con 0x7f9af40066d0 2023-10-19T10:37:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f8eda640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec005fe0 0x7f12ec04dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f8eda640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec005fe0 0x7f12ec04dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43360/0 (socket says 172.21.15.50:43360) 2023-10-19T10:37:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f8eda640 1 -- 172.21.15.50:0/4013201674 learned_addr learned my addr 172.21.15.50:0/4013201674 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.125+0000 7fe0e659c640 1 -- 172.21.15.50:0/1491844371 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d802dbd0 con 0x7fe0e80cb290 2023-10-19T10:37:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7fd456815640 1 -- 172.21.15.50:0/597884596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 msgr2=0x7fd450160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7fd456815640 1 --2- 172.21.15.50:0/597884596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450160d50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd4500ae900 tx=0x7fd44802be30 comp rx=0 tx=0).stop 2023-10-19T10:37:21.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ec055730 0x7f12ec0e0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160005990 con 0x7f815c004b60 2023-10-19T10:37:21.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160005af0 con 0x7f815c004b60 2023-10-19T10:37:21.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f816dd9b640 1 -- 172.21.15.50:0/2931137183 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160005e80 con 0x7f815c004b60 2023-10-19T10:37:21.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 shutdown_connections 2023-10-19T10:37:21.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 >> 172.21.15.50:0/3803872016 conn(0x7ff81c03aa60 msgr2=0x7ff81c03ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7fd456815640 1 -- 172.21.15.50:0/597884596 shutdown_connections 2023-10-19T10:37:21.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec04ab00 con 0x7f12ec050380 2023-10-19T10:37:21.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec04f170 con 0x7f12ec055730 2023-10-19T10:37:21.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec04e840 con 0x7f12ec005fe0 2023-10-19T10:37:21.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f3fff640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12ec050380 0x7f12ec0dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2502004316 0 0) 0x7f9aec003520 con 0x7f9af40066d0 2023-10-19T10:37:21.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 shutdown_connections 2023-10-19T10:37:21.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 >> v2:172.21.15.183:3300/0 conn(0x7f9af4053150 msgr2=0x7f9af4006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 --2- 172.21.15.50:0/128600171 >> v2:172.21.15.183:3300/0 conn(0x7f9af4053150 0x7f9af4006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 >> v1:172.21.15.183:6789/0 conn(0x7f9af4033fc0 legacy=0x7f9af4050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40537e0 con 0x7f9af40066d0 2023-10-19T10:37:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f815c004b60 msgr2=0x7f815c004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f816f59e640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f815c004b60 0x7f815c004f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8160009b70 tx=0x7f815c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f9afc04e420 con 0x7f9af40066d0 2023-10-19T10:37:21.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 >> v1:172.21.15.50:6789/0 conn(0x7f9af40066d0 legacy=0x7f9af4006b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aec003970 0x7f9aec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec0047c0 0x7f9aec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aec005180 0x7f9aec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40535d0 con 0x7f9aec0047c0 2023-10-19T10:37:21.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec00bfd0 con 0x7f9aec005180 2023-10-19T10:37:21.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afb7fe640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec0047c0 0x7f9aec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec00c190 con 0x7f9aec003970 2023-10-19T10:37:21.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9afbfff640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aec003970 0x7f9aec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7f12f96db640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ec055730 0x7f12ec0e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f3fff640 1 -- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec005fe0 msgr2=0x7f12ec04dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f3fff640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec005fe0 0x7f12ec04dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f3fff640 1 -- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ec055730 msgr2=0x7f12ec0e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f3fff640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ec055730 0x7f12ec0e0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f3fff640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec04eb20 con 0x7f12ec050380 2023-10-19T10:37:21.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f3fff640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12ec050380 0x7f12ec0dfed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12f404f3b0 tx=0x7f12f40677b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7fd456815640 1 --2- 172.21.15.50:0/597884596 >> v2:172.21.15.183:3300/0 conn(0x7fd4500ce830 0x7fd4500c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7fd456815640 1 --2- 172.21.15.50:0/597884596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450160d50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.126+0000 7fd456815640 1 -- 172.21.15.50:0/597884596 >> 172.21.15.50:0/597884596 conn(0x7fd4500ceef0 msgr2=0x7fd4500cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fd456815640 1 -- 172.21.15.50:0/597884596 shutdown_connections 2023-10-19T10:37:21.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/1491844371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e80cb290 msgr2=0x7fe0e80ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e80cb290 0x7fe0e80ad230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80afb70 tx=0x7fe0d802bb30 comp rx=0 tx=0).stop 2023-10-19T10:37:21.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/1491844371 shutdown_connections 2023-10-19T10:37:21.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/1491844371 >> v2:172.21.15.183:3300/0 conn(0x7fe0e815dfe0 0x7fe0e80cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e80ad770 0x7fe0e80cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/1491844371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e80cb290 0x7fe0e80ad230 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80afb70 tx=0x7fe0d802bb30 comp rx=0 tx=0).stop 2023-10-19T10:37:21.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/1491844371 >> 172.21.15.50:0/1491844371 conn(0x7fe0e8069e90 msgr2=0x7fe0e80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7ff82aee9640 1 -- 172.21.15.50:0/3803872016 wait complete. 2023-10-19T10:37:21.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40b7820 con 0x7f12ec050380 2023-10-19T10:37:21.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f9b008f9640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aec005180 0x7f9aec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec04ee00 con 0x7f12ec050380 2023-10-19T10:37:21.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 shutdown_connections 2023-10-19T10:37:21.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f816f59e640 1 --2- 172.21.15.50:0/2931137183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f815c004b60 0x7f815c004f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 >> 172.21.15.50:0/2931137183 conn(0x7f81700afb70 msgr2=0x7f81700aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f1cb4817640 1 Processor -- start 2023-10-19T10:37:21.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec04e320 con 0x7f12ec050380 2023-10-19T10:37:21.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7f051b59e640 1 Processor -- start 2023-10-19T10:37:21.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f051b59e640 1 -- start start 2023-10-19T10:37:21.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/1491844371 shutdown_connections 2023-10-19T10:37:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f1cb4817640 1 -- start start 2023-10-19T10:37:21.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 shutdown_connections 2023-10-19T10:37:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f40b7980 con 0x7f12ec050380 2023-10-19T10:37:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40b7cf0 con 0x7f12ec050380 2023-10-19T10:37:21.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7ff82aee9640 1 Processor -- start 2023-10-19T10:37:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/1491844371 wait complete. 2023-10-19T10:37:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cb4817640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca80e5b90 0x7f1ca80e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cb4817640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ca80387c0 0x7f1ca8177f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.127+0000 7fd456815640 1 -- 172.21.15.50:0/597884596 wait complete. 2023-10-19T10:37:21.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f9afbfff640 1 -- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aec005180 msgr2=0x7f9aec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f9afbfff640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aec005180 0x7f9aec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f9afbfff640 1 -- 172.21.15.50:0/128600171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec0047c0 msgr2=0x7f9aec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f9afbfff640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec0047c0 0x7f9aec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.128+0000 7f9afbfff640 1 -- 172.21.15.50:0/128600171 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec003520 con 0x7f9aec003970 2023-10-19T10:37:21.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7fd456815640 1 Processor -- start 2023-10-19T10:37:21.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7ff82aee9640 1 -- start start 2023-10-19T10:37:21.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051c06af40 0x7f051c14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051b59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c0c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051b59e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0c2b80 0x7f051c0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cae59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca80e5b90 0x7f1ca80e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d4005250 con 0x7f12ec050380 2023-10-19T10:37:21.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7fd456815640 1 -- start start 2023-10-19T10:37:21.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cae59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca80e5b90 0x7f1ca80e5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43378/0 (socket says 172.21.15.50:43378) 2023-10-19T10:37:21.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cb4817640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca8178470 0x7f1ca8178900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cb4817640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8039920 con 0x7f1ca80387c0 2023-10-19T10:37:21.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cb4817640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8039150 con 0x7f1ca8178470 2023-10-19T10:37:21.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cb4817640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8038c50 con 0x7f1ca80e5b90 2023-10-19T10:37:21.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cae59c640 1 -- 172.21.15.50:0/2362025927 learned_addr learned my addr 172.21.15.50:0/2362025927 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ca80387c0 0x7f1ca8177f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f1caed9d640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca8178470 0x7f1ca8178900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f0519d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c0c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f0519d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c0c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:56000/0 (socket says 172.21.15.50:56000) 2023-10-19T10:37:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0c2b80 0x7f051c0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0c2b80 0x7f051c0c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43392/0 (socket says 172.21.15.50:43392) 2023-10-19T10:37:21.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f0519d9b640 1 -- 172.21.15.50:0/2377775623 learned_addr learned my addr 172.21.15.50:0/2377775623 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c150ea0 con 0x7f051c06af40 2023-10-19T10:37:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0c3010 con 0x7f051c06e440 2023-10-19T10:37:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.129+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0c5460 con 0x7f051c0c2b80 2023-10-19T10:37:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f051a59c640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051c06af40 0x7f051c14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd456815640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450069a70 0x7fd45015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd456815640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd456815640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4500ce830 0x7fd45014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd456815640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45006ce90 con 0x7fd450069e50 2023-10-19T10:37:21.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd456815640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45006c250 con 0x7fd450069a70 2023-10-19T10:37:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd456815640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45006c4f0 con 0x7fd4500ce830 2023-10-19T10:37:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f9b008f9640 1 -- 172.21.15.50:0/128600171 reap_dead start 2023-10-19T10:37:21.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f9afbfff640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aec003970 0x7f9aec003d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9af0001520 tx=0x7f9aec0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef00aa9c6da81b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f1cadd9b640 1 -- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca80e5b90 msgr2=0x7f1ca80e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca80e5b90 0x7f1ca80e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f1cadd9b640 1 -- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca8178470 msgr2=0x7f1ca8178900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca8178470 0x7f1ca8178900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f1cadd9b640 1 -- 172.21.15.50:0/2362025927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8039360 con 0x7f1ca80387c0 2023-10-19T10:37:21.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00052d0 con 0x7f9aec003970 2023-10-19T10:37:21.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd455012640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd455012640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39062/0 (socket says 172.21.15.50:39062) 2023-10-19T10:37:21.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fd455012640 1 -- 172.21.15.50:0/3235806931 learned_addr learned my addr 172.21.15.50:0/3235806931 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0005430 con 0x7f9aec003970 2023-10-19T10:37:21.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f9afaffd640 1 -- 172.21.15.50:0/128600171 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00309a0 con 0x7f9aec003970 2023-10-19T10:37:21.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fe0ed0bd640 1 Processor -- start 2023-10-19T10:37:21.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7fe0ed0bd640 1 -- start start 2023-10-19T10:37:21.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd456014640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4500ce830 0x7fd45014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7f1cadd9b640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ca80387c0 0x7f1ca8177f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4031000 tx=0x7f1ca4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7f816f59e640 1 -- 172.21.15.50:0/2931137183 wait complete. 2023-10-19T10:37:21.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.130+0000 7ff82aee9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff814003dc0 0x7ff81c0e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff82aee9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0e5af0 0x7ff81c0e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff82aee9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff81c0d5f90 0x7ff81c0d6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff828c85640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff814003dc0 0x7ff81c0e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff828c85640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff814003dc0 0x7ff81c0e39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43412/0 (socket says 172.21.15.50:43412) 2023-10-19T10:37:21.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff828c85640 1 -- 172.21.15.50:0/4185689018 learned_addr learned my addr 172.21.15.50:0/4185689018 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0e64b0 con 0x7ff81c0d5f90 2023-10-19T10:37:21.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd456014640 1 -- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450069a70 msgr2=0x7fd45015ff10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-19T10:37:21.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd456014640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450069a70 0x7fd45015ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd456014640 1 -- 172.21.15.50:0/3235806931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 msgr2=0x7fd450151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c04da60 con 0x7ff81c0e5af0 2023-10-19T10:37:21.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0e66b0 con 0x7ff814003dc0 2023-10-19T10:37:21.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7ff823fff640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0e5af0 0x7ff81c0e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd456014640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd456014640 1 -- 172.21.15.50:0/3235806931 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c009400 con 0x7fd4500ce830 2023-10-19T10:37:21.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aec003970 msgr2=0x7f9aec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff823fff640 1 -- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff814003dc0 msgr2=0x7ff81c0e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff823fff640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff814003dc0 0x7ff81c0e39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff823fff640 1 -- 172.21.15.50:0/4185689018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff81c0d5f90 msgr2=0x7ff81c0d6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff823fff640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff81c0d5f90 0x7ff81c0d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff823fff640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0e6990 con 0x7ff81c0e5af0 2023-10-19T10:37:21.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f12f40b4050 con 0x7f12ec050380 2023-10-19T10:37:21.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aec003970 0x7f9aec003d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9af0001520 tx=0x7f9aec0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 shutdown_connections 2023-10-19T10:37:21.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f816f59e640 1 Processor -- start 2023-10-19T10:37:21.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 --2- 172.21.15.50:0/128600171 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aec003970 0x7f9aec003d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 >> 172.21.15.50:0/128600171 conn(0x7f9af4054c10 msgr2=0x7f9af4054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7fd455012640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:21.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7f051a59c640 1 -- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0c2b80 msgr2=0x7f051c0c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7f051a59c640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0c2b80 0x7f051c0c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f816f59e640 1 -- start start 2023-10-19T10:37:21.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7f051a59c640 1 -- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 msgr2=0x7f051c0c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7f051a59c640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c0c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.131+0000 7f051a59c640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051400ab80 con 0x7f051c06af40 2023-10-19T10:37:21.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff823fff640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0e5af0 0x7ff81c0e5f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff81000c780 tx=0x7ff81000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f051a59c640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051c06af40 0x7f051c14fc90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e850 tx=0x7f050c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 shutdown_connections 2023-10-19T10:37:21.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f12f17fa640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f12d806a120 0x7f12d806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7fd456014640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4500ce830 0x7fd45014ef80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e7d0 tx=0x7fd44c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7f9b010fa640 1 -- 172.21.15.50:0/128600171 wait complete. 2023-10-19T10:37:21.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7fe0ed0bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 0x7fe0e80b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0ed0bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e815dfe0 0x7fe0e81f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f12f8eda640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f12d806a120 0x7f12d806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0ed0bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e815a630 0x7fe0e814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f12f4124380 con 0x7f12ec050380 2023-10-19T10:37:21.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 0x7fe0e80b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0ed0bd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e81f6bb0 con 0x7fe0e815dfe0 2023-10-19T10:37:21.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 0x7fe0e80b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:56034/0 (socket says 172.21.15.50:56034) 2023-10-19T10:37:21.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/173236982 learned_addr learned my addr 172.21.15.50:0/173236982 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80b3dc0 con 0x7fe0e80ad770 2023-10-19T10:37:21.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100074e0 con 0x7ff81c0e5af0 2023-10-19T10:37:21.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810007660 con 0x7ff81c0e5af0 2023-10-19T10:37:21.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.132+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810004a20 con 0x7ff81c0e5af0 2023-10-19T10:37:21.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c0e6c70 con 0x7ff81c0e5af0 2023-10-19T10:37:21.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c0e44d0 con 0x7ff81c0e5af0 2023-10-19T10:37:21.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80c6590 con 0x7fe0e815a630 2023-10-19T10:37:21.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e759e640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e815a630 0x7fe0e814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0dffff640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e815dfe0 0x7fe0e81f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00eee0 con 0x7fd4500ce830 2023-10-19T10:37:21.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c004020 con 0x7fd4500ce830 2023-10-19T10:37:21.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0043b0 con 0x7fd4500ce830 2023-10-19T10:37:21.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f816f59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c004b60 0x7f8170156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f816f59e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81701585b0 0x7f8170158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f816f59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 0x7f81700c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f816f59e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170157d70 con 0x7f81700c6de0 2023-10-19T10:37:21.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f816f59e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170158f70 con 0x7f815c004b60 2023-10-19T10:37:21.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7f816f59e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8170156d60 con 0x7f81701585b0 2023-10-19T10:37:21.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd448009a70 con 0x7fd4500ce830 2023-10-19T10:37:21.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45006cab0 con 0x7fd4500ce830 2023-10-19T10:37:21.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c04dc10 con 0x7ff81c0e5af0 2023-10-19T10:37:21.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 0x7f81700c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 0x7f81700c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39080/0 (socket says 172.21.15.50:39080) 2023-10-19T10:37:21.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 -- 172.21.15.50:0/101675680 learned_addr learned my addr 172.21.15.50:0/101675680 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f8167fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81701585b0 0x7f8170158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 -- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81701585b0 msgr2=0x7f8170158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81701585b0 0x7f8170158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 -- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c004b60 msgr2=0x7f8170156590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-19T10:37:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c004b60 0x7f8170156590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7f816ed9d640 1 -- 172.21.15.50:0/101675680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81701578e0 con 0x7f81700c6de0 2023-10-19T10:37:21.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7f816ed9d640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 0x7f81700c7260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81701618e0 tx=0x7f816800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e815a630 msgr2=0x7fe0e814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e815a630 0x7fe0e814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/173236982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e815dfe0 msgr2=0x7fe0e81f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e815dfe0 0x7fe0e81f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e6d9d640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8009400 con 0x7fe0e80ad770 2023-10-19T10:37:21.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.133+0000 7fe0e759e640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e815a630 0x7fe0e814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:21.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7fe0e6d9d640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 0x7fe0e80b2030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0d802c070 tx=0x7fe0d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80359e0 con 0x7fe0e80ad770 2023-10-19T10:37:21.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d8035b60 con 0x7fe0e80ad770 2023-10-19T10:37:21.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.134+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d803e420 con 0x7fe0e80ad770 2023-10-19T10:37:21.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7f12f8eda640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f12d806a120 0x7f12d806c5e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0dfd20 tx=0x7f12e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd450066900 con 0x7fd4500ce830 2023-10-19T10:37:21.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7f9b010fa640 1 Processor -- start 2023-10-19T10:37:21.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e80c6870 con 0x7fe0e80ad770 2023-10-19T10:37:21.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e80c6e30 con 0x7fe0e80ad770 2023-10-19T10:37:21.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff810004b80 con 0x7ff81c0e5af0 2023-10-19T10:37:21.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.135+0000 7f9b010fa640 1 -- start start 2023-10-19T10:37:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.136+0000 7ff8217fa640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff80806a050 0x7ff80806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.136+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff810087b40 con 0x7ff81c0e5af0 2023-10-19T10:37:21.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.136+0000 7ff828c85640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff80806a050 0x7ff80806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.136+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd44c005990 con 0x7fd4500ce830 2023-10-19T10:37:21.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.136+0000 7ff828c85640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff80806a050 0x7ff80806c510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff824078c80 tx=0x7ff824061600 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.136+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e8073990 con 0x7fe0e80ad770 2023-10-19T10:37:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b010fa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 0x7f9af40d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7fd4467fc640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd43406a0a0 0x7fd43406c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b010fa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af40d44b0 0x7f9af40d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b010fa640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af40d7740 0x7f9af4036c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b010fa640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40d7260 con 0x7f9aec003970 2023-10-19T10:37:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b010fa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af404b570 con 0x7f9af40d44b0 2023-10-19T10:37:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b010fa640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40d8870 con 0x7f9af40d7740 2023-10-19T10:37:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 0x7f9af40d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd44c088a40 con 0x7fd4500ce830 2023-10-19T10:37:21.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 0x7f9af40d82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39086/0 (socket says 172.21.15.50:39086) 2023-10-19T10:37:21.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 -- 172.21.15.50:0/2618568830 learned_addr learned my addr 172.21.15.50:0/2618568830 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:21.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7fd455813640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd43406a0a0 0x7fd43406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afb7fe640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af40d44b0 0x7f9af40d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9b008f9640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af40d7740 0x7f9af4036c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe0d803e580 con 0x7fe0e80ad770 2023-10-19T10:37:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 -- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af40d7740 msgr2=0x7f9af4036c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af40d7740 0x7f9af4036c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 -- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af40d44b0 msgr2=0x7f9af40d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af40d44b0 0x7f9af40d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 -- 172.21.15.50:0/2618568830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40d8b50 con 0x7f9aec003970 2023-10-19T10:37:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.137+0000 7f9afbfff640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 0x7f9af40d82b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9af0001640 tx=0x7f9af0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.138+0000 7fe0df7fe640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe0c807b060 0x7fe0c807d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.138+0000 7fd455813640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd43406a0a0 0x7fd43406c560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd440000f60 tx=0x7fd4400010a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.138+0000 7fe0dffff640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe0c807b060 0x7fe0c807d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.138+0000 7fe0dffff640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe0c807b060 0x7fe0c807d520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0e814db90 tx=0x7fe0d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f12f40ef760 con 0x7f12ec050380 2023-10-19T10:37:21.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c00c940 con 0x7f051c06af40 2023-10-19T10:37:21.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4001630 con 0x7f1ca80387c0 2023-10-19T10:37:21.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca403a070 con 0x7f1ca80387c0 2023-10-19T10:37:21.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00052d0 con 0x7f9aec003970 2023-10-19T10:37:21.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af40d3b20 con 0x7f9aec003970 2023-10-19T10:37:21.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af404b190 con 0x7f9aec003970 2023-10-19T10:37:21.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca8039640 con 0x7f1ca80387c0 2023-10-19T10:37:21.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c00caa0 con 0x7f051c06af40 2023-10-19T10:37:21.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168003ba0 con 0x7f81700c6de0 2023-10-19T10:37:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8168003d00 con 0x7f81700c6de0 2023-10-19T10:37:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168005310 con 0x7f81700c6de0 2023-10-19T10:37:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c00ce10 con 0x7f051c06af40 2023-10-19T10:37:21.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051c150500 con 0x7f051c06af40 2023-10-19T10:37:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f816f59e640 1 -- 172.21.15.50:0/101675680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81700b1a10 con 0x7f81700c6de0 2023-10-19T10:37:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c150ac0 con 0x7f051c06af40 2023-10-19T10:37:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca80da3e0 con 0x7f1ca80387c0 2023-10-19T10:37:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00035f0 con 0x7f9aec003970 2023-10-19T10:37:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00039c0 con 0x7f9aec003970 2023-10-19T10:37:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.139+0000 7f816f59e640 1 -- 172.21.15.50:0/101675680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170157290 con 0x7f81700c6de0 2023-10-19T10:37:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.140+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4007270 con 0x7f1ca80387c0 2023-10-19T10:37:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.140+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af404b190 con 0x7f9aec003970 2023-10-19T10:37:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.141+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051c150fe0 con 0x7f051c06af40 2023-10-19T10:37:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.141+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8170163ed0 con 0x7f81700c6de0 2023-10-19T10:37:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.141+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd44c053ed0 con 0x7fd4500ce830 2023-10-19T10:37:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.141+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff81c04dc10 con 0x7ff81c0e5af0 2023-10-19T10:37:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.141+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe0d8031070 con 0x7fe0e80ad770 2023-10-19T10:37:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.141+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c94005250 con 0x7f1ca80387c0 2023-10-19T10:37:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.143+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1ca4032070 con 0x7f1ca80387c0 2023-10-19T10:37:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.143+0000 7f1c9effd640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c9006a020 0x7f1c9006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.143+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9af0003b20 con 0x7f9aec003970 2023-10-19T10:37:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f9af8ff9640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ae006a050 0x7f9ae006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f9afb7fe640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ae006a050 0x7f9ae006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f1cae59c640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c9006a020 0x7f1c9006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8168006b30 con 0x7f81700c6de0 2023-10-19T10:37:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f8166ffd640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f815006a0a0 0x7f815006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f050c013470 con 0x7f051c06af40 2023-10-19T10:37:21.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.144+0000 7f050affd640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f04fc06a050 0x7f04fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:21.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f816e59c640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f815006a0a0 0x7f815006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f0519d9b640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f04fc06a050 0x7f04fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f9afb7fe640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ae006a050 0x7f9ae006c510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9af40d4890 tx=0x7f9ae8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f1cae59c640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c9006a020 0x7f1c9006c4e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1cb004ee40 tx=0x7f1cb0068830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1ca407cc00 con 0x7f1ca80387c0 2023-10-19T10:37:21.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8168058b30 con 0x7f81700c6de0 2023-10-19T10:37:21.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f050c05be80 con 0x7f051c06af40 2023-10-19T10:37:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f0519d9b640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f04fc06a050 0x7f04fc06c510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f051c1f9510 tx=0x7f0510009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.145+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9af0081f40 con 0x7f9aec003970 2023-10-19T10:37:21.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.146+0000 7f816e59c640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f815006a0a0 0x7f815006c560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8160035000 tx=0x7f8160030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.146+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe0d8075bb0 con 0x7fe0e80ad770 2023-10-19T10:37:21.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.151+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1ca4075230 con 0x7f1ca80387c0 2023-10-19T10:37:21.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.155+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f81680547e0 con 0x7f81700c6de0 2023-10-19T10:37:21.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.167+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9af007a570 con 0x7f9aec003970 2023-10-19T10:37:21.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.176+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f050c0544b0 con 0x7f051c06af40 2023-10-19T10:37:21.531 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2023-10-19T10:37:21.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.525+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f12d40057d0 con 0x7f12ec050380 2023-10-19T10:37:21.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.525+0000 7f12f17fa640 1 -- 172.21.15.50:0/4013201674 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f12f40b2090 con 0x7f12ec050380 2023-10-19T10:37:21.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.528+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f12d806a120 msgr2=0x7f12d806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.528+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f12d806a120 0x7f12d806c5e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0dfd20 tx=0x7f12e0009040 comp rx=0 tx=0).stop 2023-10-19T10:37:21.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.528+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12ec050380 msgr2=0x7f12ec0dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.528+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12ec050380 0x7f12ec0dfed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12f404f3b0 tx=0x7f12f40677b0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 shutdown_connections 2023-10-19T10:37:21.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f12d806a120 0x7f12d806c5e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ec055730 0x7f12ec0e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12ec050380 0x7f12ec0dfed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 --2- 172.21.15.50:0/4013201674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f12ec005fe0 0x7f12ec04dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 >> 172.21.15.50:0/4013201674 conn(0x7f12ec054c10 msgr2=0x7f12ec0ddf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 shutdown_connections 2023-10-19T10:37:21.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.529+0000 7f12f9edc640 1 -- 172.21.15.50:0/4013201674 wait complete. 2023-10-19T10:37:21.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-10-19T10:37:21.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.551+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff81c04dc10 con 0x7ff81c0e5af0 2023-10-19T10:37:21.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.551+0000 7ff8217fa640 1 -- 172.21.15.50:0/4185689018 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff810018020 con 0x7ff81c0e5af0 2023-10-19T10:37:21.554 INFO:teuthology.orchestra.run.smithi050.stdout:55834574856 2023-10-19T10:37:21.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.554+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff80806a050 msgr2=0x7ff80806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.554+0000 7ff82aee9640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff80806a050 0x7ff80806c510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff824078c80 tx=0x7ff824061600 comp rx=0 tx=0).stop 2023-10-19T10:37:21.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.554+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0e5af0 msgr2=0x7ff81c0e5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.554+0000 7ff82aee9640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0e5af0 0x7ff81c0e5f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff81000c780 tx=0x7ff81000cc40 comp rx=0 tx=0).stop 2023-10-19T10:37:21.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 shutdown_connections 2023-10-19T10:37:21.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff80806a050 0x7ff80806c510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff81c0d5f90 0x7ff81c0d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff81c0e5af0 0x7ff81c0e5f70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 --2- 172.21.15.50:0/4185689018 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff814003dc0 0x7ff81c0e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 >> 172.21.15.50:0/4185689018 conn(0x7ff81c03aa60 msgr2=0x7ff81c0e8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 shutdown_connections 2023-10-19T10:37:21.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.555+0000 7ff82aee9640 1 -- 172.21.15.50:0/4185689018 wait complete. 2023-10-19T10:37:21.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.558+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd45006c630 con 0x7fd4500ce830 2023-10-19T10:37:21.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.558+0000 7fd4467fc640 1 -- 172.21.15.50:0/3235806931 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd45006c630 con 0x7fd4500ce830 2023-10-19T10:37:21.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.558+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9af404b190 con 0x7f9aec003970 2023-10-19T10:37:21.562 INFO:teuthology.orchestra.run.smithi050.stdout:55834574856 2023-10-19T10:37:21.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.559+0000 7f9af8ff9640 1 -- 172.21.15.50:0/2618568830 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9af0081090 con 0x7f9aec003970 2023-10-19T10:37:21.562 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2023-10-19T10:37:21.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.561+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ae006a050 msgr2=0x7f9ae006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.561+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1c94005500 con 0x7f1ca80387c0 2023-10-19T10:37:21.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.561+0000 7f9b010fa640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ae006a050 0x7f9ae006c510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9af40d4890 tx=0x7f9ae8008040 comp rx=0 tx=0).stop 2023-10-19T10:37:21.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 msgr2=0x7f9af40d82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 0x7f9af40d82b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9af0001640 tx=0x7f9af0001120 comp rx=0 tx=0).stop 2023-10-19T10:37:21.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f1c9effd640 1 -- 172.21.15.50:0/2362025927 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ca4045070 con 0x7f1ca80387c0 2023-10-19T10:37:21.566 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:21.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 shutdown_connections 2023-10-19T10:37:21.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ae006a050 0x7f9ae006c510 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af40d7740 0x7f9af4036c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af40d44b0 0x7f9af40d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 --2- 172.21.15.50:0/2618568830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec003970 0x7f9af40d82b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd43406a0a0 msgr2=0x7fd43406c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd43406a0a0 0x7fd43406c560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd440000f60 tx=0x7fd4400010a0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4500ce830 msgr2=0x7fd45014ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 >> 172.21.15.50:0/2618568830 conn(0x7f9af4054c10 msgr2=0x7f9af40d49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4500ce830 0x7fd45014ef80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e7d0 tx=0x7fd44c00ec90 comp rx=0 tx=0).stop 2023-10-19T10:37:21.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 shutdown_connections 2023-10-19T10:37:21.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd43406a0a0 0x7fd43406c560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 shutdown_connections 2023-10-19T10:37:21.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd4500ce830 0x7fd45014ef80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd450069e50 0x7fd450151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 --2- 172.21.15.50:0/3235806931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd450069a70 0x7fd45015ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 >> 172.21.15.50:0/3235806931 conn(0x7fd4500ceef0 msgr2=0x7fd45006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 shutdown_connections 2023-10-19T10:37:21.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7f9b010fa640 1 -- 172.21.15.50:0/2618568830 wait complete. 2023-10-19T10:37:21.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.562+0000 7fd456815640 1 -- 172.21.15.50:0/3235806931 wait complete. 2023-10-19T10:37:21.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.563+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe0e80b0940 con 0x7fe0e80ad770 2023-10-19T10:37:21.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.563+0000 7fe0df7fe640 1 -- 172.21.15.50:0/173236982 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0d8045020 con 0x7fe0e80ad770 2023-10-19T10:37:21.575 INFO:teuthology.orchestra.run.smithi050.stdout:60129542154 2023-10-19T10:37:21.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.565+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c9006a020 msgr2=0x7f1c9006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.565+0000 7f1cb4817640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c9006a020 0x7f1c9006c4e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1cb004ee40 tx=0x7f1cb0068830 comp rx=0 tx=0).stop 2023-10-19T10:37:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ca80387c0 msgr2=0x7f1ca8177f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ca80387c0 0x7f1ca8177f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4031000 tx=0x7f1ca4001930 comp rx=0 tx=0).stop 2023-10-19T10:37:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 shutdown_connections 2023-10-19T10:37:21.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c9006a020 0x7f1c9006c4e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca8178470 0x7f1ca8178900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ca80387c0 0x7f1ca8177f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 --2- 172.21.15.50:0/2362025927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ca80e5b90 0x7f1ca80e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 >> 172.21.15.50:0/2362025927 conn(0x7f1ca8055c00 msgr2=0x7f1ca80e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 shutdown_connections 2023-10-19T10:37:21.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.566+0000 7f1cb4817640 1 -- 172.21.15.50:0/2362025927 wait complete. 2023-10-19T10:37:21.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe0c807b060 msgr2=0x7fe0c807d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe0c807b060 0x7fe0c807d520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0e814db90 tx=0x7fe0d0008040 comp rx=0 tx=0).stop 2023-10-19T10:37:21.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 msgr2=0x7fe0e80b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 0x7fe0e80b2030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0d802c070 tx=0x7fe0d80046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 shutdown_connections 2023-10-19T10:37:21.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe0c807b060 0x7fe0c807d520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0e815a630 0x7fe0e814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0e815dfe0 0x7fe0e81f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f817004e420 con 0x7f81700c6de0 2023-10-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 --2- 172.21.15.50:0/173236982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0e80ad770 0x7fe0e80b2030 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0d802c070 tx=0x7fe0d80046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 >> 172.21.15.50:0/173236982 conn(0x7fe0e8069e90 msgr2=0x7fe0e8070170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 shutdown_connections 2023-10-19T10:37:21.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.567+0000 7fe0ed0bd640 1 -- 172.21.15.50:0/173236982 wait complete. 2023-10-19T10:37:21.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.568+0000 7f8166ffd640 1 -- 172.21.15.50:0/101675680 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8168057e00 con 0x7f81700c6de0 2023-10-19T10:37:21.583 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2023-10-19T10:37:21.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.571+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f815006a0a0 msgr2=0x7f815006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.571+0000 7f8164ff9640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f815006a0a0 0x7f815006c560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8160035000 tx=0x7f8160030000 comp rx=0 tx=0).stop 2023-10-19T10:37:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.571+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 msgr2=0x7f81700c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.571+0000 7f8164ff9640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 0x7f81700c7260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81701618e0 tx=0x7f816800ef00 comp rx=0 tx=0).stop 2023-10-19T10:37:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.571+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 shutdown_connections 2023-10-19T10:37:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f815006a0a0 0x7f815006c560 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81700c6de0 0x7f81700c7260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81701585b0 0x7f8170158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 --2- 172.21.15.50:0/101675680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c004b60 0x7f8170156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 >> 172.21.15.50:0/101675680 conn(0x7f81700afb70 msgr2=0x7f81700caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 shutdown_connections 2023-10-19T10:37:21.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.572+0000 7f8164ff9640 1 -- 172.21.15.50:0/101675680 wait complete. 2023-10-19T10:37:21.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-10-19T10:37:21.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-10-19T10:37:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.582+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f051c066e80 con 0x7f051c06af40 2023-10-19T10:37:21.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.583+0000 7f050affd640 1 -- 172.21.15.50:0/2377775623 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f050c021090 con 0x7f051c06af40 2023-10-19T10:37:21.590 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:21.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.7 2023-10-19T10:37:21.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2023-10-19T10:37:21.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f04fc06a050 msgr2=0x7f04fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f04fc06a050 0x7f04fc06c510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f051c1f9510 tx=0x7f0510009040 comp rx=0 tx=0).stop 2023-10-19T10:37:21.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051c06af40 msgr2=0x7f051c14fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:21.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051c06af40 0x7f051c14fc90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e850 tx=0x7f050c00ed10 comp rx=0 tx=0).stop 2023-10-19T10:37:21.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 shutdown_connections 2023-10-19T10:37:21.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f04fc06a050 0x7f04fc06c510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0c2b80 0x7f051c0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c06e440 0x7f051c0c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 --2- 172.21.15.50:0/2377775623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f051c06af40 0x7f051c14fc90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:21.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 >> 172.21.15.50:0/2377775623 conn(0x7f051c0b01d0 msgr2=0x7f051c159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:21.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 shutdown_connections 2023-10-19T10:37:21.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:21.586+0000 7f051b59e640 1 -- 172.21.15.50:0/2377775623 wait complete. 2023-10-19T10:37:21.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2023-10-19T10:37:21.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-10-19T10:37:21.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2023-10-19T10:37:22.548 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-19T10:37:22.588 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-19T10:37:22.590 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-19T10:37:22.592 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-19T10:37:22.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-19T10:37:22.597 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-19T10:37:22.598 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-19T10:37:22.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-19T10:37:22.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.716+0000 7f40df999640 1 Processor -- start 2023-10-19T10:37:22.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.717+0000 7f40df999640 1 -- start start 2023-10-19T10:37:22.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.718+0000 7f40df999640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f40d8159550 0x7f40d80b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.718+0000 7f40df999640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8070e90 0x7f40d80cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.718+0000 7f40df999640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d80718a0 con 0x7f40d80ccd10 2023-10-19T10:37:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.718+0000 7f40df999640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d806e610 con 0x7f40d8159550 2023-10-19T10:37:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.718+0000 7f40df999640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80ca8d0 con 0x7f40d8070e90 2023-10-19T10:37:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40ddf36640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8070e90 0x7f40d80cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40dcf34640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f40d80ccd10 0x7f40d8070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41932/0 (socket says 172.21.15.50:41932) 2023-10-19T10:37:22.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40dcf34640 1 -- 172.21.15.50:0/2206024116 learned_addr learned my addr 172.21.15.50:0/2206024116 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:22.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40ddf36640 1 -- 172.21.15.50:0/2206024116 >> v2:172.21.15.50:3300/0 conn(0x7f40d8159550 msgr2=0x7f40d80b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40dd735640 1 --2- 172.21.15.50:0/2206024116 >> v2:172.21.15.50:3300/0 conn(0x7f40d8159550 0x7f40d80b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40ddf36640 1 --2- 172.21.15.50:0/2206024116 >> v2:172.21.15.50:3300/0 conn(0x7f40d8159550 0x7f40d80b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40ddf36640 1 -- 172.21.15.50:0/2206024116 >> v1:172.21.15.183:6789/0 conn(0x7f40d80ccd10 legacy=0x7f40d8070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.719+0000 7f40ddf36640 1 -- 172.21.15.50:0/2206024116 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d806e860 con 0x7f40d8070e90 2023-10-19T10:37:22.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.720+0000 7f40dd735640 1 --2- 172.21.15.50:0/2206024116 >> v2:172.21.15.50:3300/0 conn(0x7f40d8159550 0x7f40d80b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:22.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.721+0000 7f40ddf36640 1 --2- 172.21.15.50:0/2206024116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8070e90 0x7f40d80cad60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009d90 tx=0x7f40d002bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1762fcba6a1a622 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.722+0000 7f40cffff640 1 -- 172.21.15.50:0/2206024116 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d002ca30 con 0x7f40d8070e90 2023-10-19T10:37:22.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.722+0000 7f40cffff640 1 -- 172.21.15.50:0/2206024116 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d002cb90 con 0x7f40d8070e90 2023-10-19T10:37:22.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.723+0000 7f40df999640 1 -- 172.21.15.50:0/2206024116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8070e90 msgr2=0x7f40d80cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.723+0000 7f40df999640 1 --2- 172.21.15.50:0/2206024116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8070e90 0x7f40d80cad60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009d90 tx=0x7f40d002bd00 comp rx=0 tx=0).stop 2023-10-19T10:37:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.724+0000 7f40df999640 1 -- 172.21.15.50:0/2206024116 shutdown_connections 2023-10-19T10:37:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.724+0000 7f40df999640 1 --2- 172.21.15.50:0/2206024116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8070e90 0x7f40d80cad60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.724+0000 7f40df999640 1 --2- 172.21.15.50:0/2206024116 >> v2:172.21.15.50:3300/0 conn(0x7f40d8159550 0x7f40d80b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.724+0000 7f40df999640 1 -- 172.21.15.50:0/2206024116 >> 172.21.15.50:0/2206024116 conn(0x7f40d80cbbc0 msgr2=0x7f40d80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.724+0000 7f40df999640 1 -- 172.21.15.50:0/2206024116 shutdown_connections 2023-10-19T10:37:22.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.724+0000 7f40df999640 1 -- 172.21.15.50:0/2206024116 wait complete. 2023-10-19T10:37:22.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.726+0000 7f40df999640 1 Processor -- start 2023-10-19T10:37:22.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.726+0000 7f40df999640 1 -- start start 2023-10-19T10:37:22.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.727+0000 7f40df999640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 0x7f40d8089560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.727+0000 7f40df999640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80ccd10 0x7f40d815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.728+0000 7f40dd735640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 0x7f40d8089560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.728+0000 7f40dd735640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 0x7f40d8089560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39104/0 (socket says 172.21.15.50:39104) 2023-10-19T10:37:22.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40dcf34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80ccd10 0x7f40d815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40dcf34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80ccd10 0x7f40d815b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:56078/0 (socket says 172.21.15.50:56078) 2023-10-19T10:37:22.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40dcf34640 1 -- 172.21.15.50:0/921233357 learned_addr learned my addr 172.21.15.50:0/921233357 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40df999640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8159550 0x7f40d8153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c6fd0 con 0x7f40d8070e90 2023-10-19T10:37:22.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c4150 con 0x7f40d80ccd10 2023-10-19T10:37:22.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d815c030 con 0x7f40d8159550 2023-10-19T10:37:22.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.729+0000 7f40ddf36640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8159550 0x7f40d8153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.730+0000 7f40dd735640 1 -- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8159550 msgr2=0x7f40d8153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.730+0000 7f40dd735640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8159550 0x7f40d8153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.730+0000 7f40dd735640 1 -- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80ccd10 msgr2=0x7f40d815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.730+0000 7f40dd735640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80ccd10 0x7f40d815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.730+0000 7f40dd735640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d0009a70 con 0x7f40d8070e90 2023-10-19T10:37:22.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.731+0000 7f40dd735640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 0x7f40d8089560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40c800c7c0 tx=0x7f40c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.732+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8007820 con 0x7f40d8070e90 2023-10-19T10:37:22.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.732+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d80c6210 con 0x7f40d8070e90 2023-10-19T10:37:22.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.732+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d80c67d0 con 0x7f40d8070e90 2023-10-19T10:37:22.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.734+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d8073990 con 0x7f40d8070e90 2023-10-19T10:37:22.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.743+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c80079a0 con 0x7f40d8070e90 2023-10-19T10:37:22.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.743+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8007d50 con 0x7f40d8070e90 2023-10-19T10:37:22.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.744+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f40c80115d0 con 0x7f40d8070e90 2023-10-19T10:37:22.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.744+0000 7f40cdffb640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f40bc06a1a0 0x7f40bc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.745+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f40c808c710 con 0x7f40d8070e90 2023-10-19T10:37:22.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.745+0000 7f40dcf34640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f40bc06a1a0 0x7f40bc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.745+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f40c8057b10 con 0x7f40d8070e90 2023-10-19T10:37:22.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.746+0000 7f40dcf34640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f40bc06a1a0 0x7f40bc06c660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f40d81515b0 tx=0x7f40d4007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.760+0000 7f2f2412e640 1 Processor -- start 2023-10-19T10:37:22.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.760+0000 7f2f2412e640 1 -- start start 2023-10-19T10:37:22.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f2412e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c069800 0x7f2f1c161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f2412e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2f1c161b50 0x7f2f1c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f2412e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0701d0 con 0x7f2f1c0cc550 2023-10-19T10:37:22.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f2412e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c06c0a0 con 0x7f2f1c161b50 2023-10-19T10:37:22.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f2412e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0704b0 con 0x7f2f1c069800 2023-10-19T10:37:22.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f21eca640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2f1c0cc550 0x7f2f1c0cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33588/0 (socket says 172.21.15.50:33588) 2023-10-19T10:37:22.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.762+0000 7f2f21eca640 1 -- 172.21.15.50:0/2782575450 learned_addr learned my addr 172.21.15.50:0/2782575450 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:22.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.763+0000 7f2f216c9640 1 --2- 172.21.15.50:0/2782575450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c069800 0x7f2f1c161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.763+0000 7f2f216c9640 1 -- 172.21.15.50:0/2782575450 >> v2:172.21.15.183:3300/0 conn(0x7f2f1c161b50 msgr2=0x7f2f1c06c4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-19T10:37:22.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.763+0000 7f2f216c9640 1 --2- 172.21.15.50:0/2782575450 >> v2:172.21.15.183:3300/0 conn(0x7f2f1c161b50 0x7f2f1c06c4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.763+0000 7f2f216c9640 1 -- 172.21.15.50:0/2782575450 >> v1:172.21.15.50:6789/0 conn(0x7f2f1c0cc550 legacy=0x7f2f1c0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.763+0000 7f2f216c9640 1 -- 172.21.15.50:0/2782575450 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c070790 con 0x7f2f1c069800 2023-10-19T10:37:22.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.765+0000 7f2f216c9640 1 --2- 172.21.15.50:0/2782575450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c069800 0x7f2f1c161610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f180097a0 tx=0x7f2f1802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4348246edd805f0d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.765+0000 7f2f20ec8640 1 -- 172.21.15.50:0/2782575450 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18031640 con 0x7f2f1c069800 2023-10-19T10:37:22.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.765+0000 7f2f20ec8640 1 -- 172.21.15.50:0/2782575450 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f180317a0 con 0x7f2f1c069800 2023-10-19T10:37:22.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.765+0000 7f2f20ec8640 1 -- 172.21.15.50:0/2782575450 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18031b70 con 0x7f2f1c069800 2023-10-19T10:37:22.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.766+0000 7f2f2412e640 1 -- 172.21.15.50:0/2782575450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c069800 msgr2=0x7f2f1c161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.766+0000 7f2f2412e640 1 --2- 172.21.15.50:0/2782575450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c069800 0x7f2f1c161610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f180097a0 tx=0x7f2f1802b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.767+0000 7f2f2412e640 1 -- 172.21.15.50:0/2782575450 shutdown_connections 2023-10-19T10:37:22.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.767+0000 7f2f2412e640 1 --2- 172.21.15.50:0/2782575450 >> v2:172.21.15.183:3300/0 conn(0x7f2f1c161b50 0x7f2f1c06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.767+0000 7f2f2412e640 1 --2- 172.21.15.50:0/2782575450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c069800 0x7f2f1c161610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.767+0000 7f2f2412e640 1 -- 172.21.15.50:0/2782575450 >> 172.21.15.50:0/2782575450 conn(0x7f2f1c0ce980 msgr2=0x7f2f1c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.767+0000 7f2f2412e640 1 -- 172.21.15.50:0/2782575450 shutdown_connections 2023-10-19T10:37:22.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.767+0000 7f2f2412e640 1 -- 172.21.15.50:0/2782575450 wait complete. 2023-10-19T10:37:22.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.769+0000 7f2f2412e640 1 Processor -- start 2023-10-19T10:37:22.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.769+0000 7f2f2412e640 1 -- start start 2023-10-19T10:37:22.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.770+0000 7f2f2412e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 0x7f2f1c14c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 0x7f2f1c14c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 0x7f2f1c14c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39118/0 (socket says 172.21.15.50:39118) 2023-10-19T10:37:22.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f6b77a84640 1 Processor -- start 2023-10-19T10:37:22.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.772+0000 7f6b77a84640 1 -- start start 2023-10-19T10:37:22.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 -- 172.21.15.50:0/825486269 learned_addr learned my addr 172.21.15.50:0/825486269 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f2412e640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1c0cc550 0x7f2f1c14cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f2412e640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c161b50 0x7f2f1c149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f2412e640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b1b60 con 0x7f2f1c069800 2023-10-19T10:37:22.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f2412e640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c127af0 con 0x7f2f1c0cc550 2023-10-19T10:37:22.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f2412e640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c127dd0 con 0x7f2f1c161b50 2023-10-19T10:37:22.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 -- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c161b50 msgr2=0x7f2f1c149e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-19T10:37:22.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c161b50 0x7f2f1c149e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 -- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1c0cc550 msgr2=0x7f2f1c14cc60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-19T10:37:22.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1c0cc550 0x7f2f1c14cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.771+0000 7f2f21eca640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18009400 con 0x7f2f1c069800 2023-10-19T10:37:22.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.772+0000 7f2f21eca640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 0x7f2f1c14c720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400cda0 tx=0x7f2f1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400ad40 con 0x7f2f1c069800 2023-10-19T10:37:22.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f2f2412e640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c1280b0 con 0x7f2f1c069800 2023-10-19T10:37:22.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14012400 con 0x7f2f1c069800 2023-10-19T10:37:22.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400aea0 con 0x7f2f1c069800 2023-10-19T10:37:22.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.775+0000 7f0179711640 1 Processor -- start 2023-10-19T10:37:22.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f6b77a84640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700ccd10 0x7f6b700cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f6b77a84640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6b700c9dc0 0x7f6b700ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f6b77a84640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700cd6b0 con 0x7f6b700c22e0 2023-10-19T10:37:22.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f6b77a84640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b700c26c0 con 0x7f6b700c9dc0 2023-10-19T10:37:22.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f6b77a84640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70071850 con 0x7f6b700ccd10 2023-10-19T10:37:22.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.774+0000 7f6b76021640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6b700c9dc0 0x7f6b700ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.774+0000 7f6b76021640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6b700c9dc0 0x7f6b700ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39128/0 (socket says 172.21.15.50:39128) 2023-10-19T10:37:22.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.774+0000 7f6b76021640 1 -- 172.21.15.50:0/2280204674 learned_addr learned my addr 172.21.15.50:0/2280204674 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.774+0000 7f6b7501f640 1 --2- 172.21.15.50:0/2280204674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700ccd10 0x7f6b700cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.775+0000 7f0179711640 1 -- start start 2023-10-19T10:37:22.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.775+0000 7f6b7501f640 1 -- 172.21.15.50:0/2280204674 >> v2:172.21.15.50:3300/0 conn(0x7f6b700c9dc0 msgr2=0x7f6b700ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.775+0000 7f6b7501f640 1 --2- 172.21.15.50:0/2280204674 >> v2:172.21.15.50:3300/0 conn(0x7f6b700c9dc0 0x7f6b700ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.775+0000 7f6b7501f640 1 -- 172.21.15.50:0/2280204674 >> v1:172.21.15.50:6789/0 conn(0x7f6b700c22e0 legacy=0x7f6b70071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.775+0000 7f6b7501f640 1 -- 172.21.15.50:0/2280204674 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7006a380 con 0x7f6b700ccd10 2023-10-19T10:37:22.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.773+0000 7f2f2412e640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c087af0 con 0x7f2f1c069800 2023-10-19T10:37:22.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc005250 con 0x7f2f1c069800 2023-10-19T10:37:22.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0179711640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 0x7f01740cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0179711640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017406cad0 con 0x7f017415cd10 2023-10-19T10:37:22.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0179711640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017406cdb0 con 0x7f01740ccf50 2023-10-19T10:37:22.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0179711640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017406d750 con 0x7f0174159040 2023-10-19T10:37:22.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0172ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 0x7f01740cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0172ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 0x7f01740cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43476/0 (socket says 172.21.15.50:43476) 2023-10-19T10:37:22.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f0172ffd640 1 -- 172.21.15.50:0/2785442465 learned_addr learned my addr 172.21.15.50:0/2785442465 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.777+0000 7f0172ffd640 1 -- 172.21.15.50:0/2785442465 >> v1:172.21.15.183:6789/0 conn(0x7f01740ccf50 legacy=0x7f01740704b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-19T10:37:22.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.777+0000 7f0172ffd640 1 -- 172.21.15.50:0/2785442465 >> v1:172.21.15.50:6789/0 conn(0x7f017415cd10 legacy=0x7f0174070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.777+0000 7f0172ffd640 1 -- 172.21.15.50:0/2785442465 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017406d090 con 0x7f0174159040 2023-10-19T10:37:22.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f0172ffd640 1 --2- 172.21.15.50:0/2785442465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 0x7f01740cd630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f016400de80 tx=0x7f016402feb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b977f78e00af0e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.776+0000 7f6b7501f640 1 --2- 172.21.15.50:0/2280204674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700ccd10 0x7f6b700cd170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097a0 tx=0x7f6b6002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fbb7375d85cdcd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.777+0000 7f6b7481e640 1 -- 172.21.15.50:0/2280204674 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60031780 con 0x7f6b700ccd10 2023-10-19T10:37:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.777+0000 7f6b7481e640 1 -- 172.21.15.50:0/2280204674 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600318e0 con 0x7f6b700ccd10 2023-10-19T10:37:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f6b77a84640 1 -- 172.21.15.50:0/2280204674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700ccd10 msgr2=0x7f6b700cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f6b77a84640 1 --2- 172.21.15.50:0/2280204674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700ccd10 0x7f6b700cd170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097a0 tx=0x7f6b6002b820 comp rx=0 tx=0).stop 2023-10-19T10:37:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f6b77a84640 1 -- 172.21.15.50:0/2280204674 shutdown_connections 2023-10-19T10:37:22.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f6b77a84640 1 --2- 172.21.15.50:0/2280204674 >> v2:172.21.15.50:3300/0 conn(0x7f6b700c9dc0 0x7f6b700ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f6b77a84640 1 --2- 172.21.15.50:0/2280204674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700ccd10 0x7f6b700cd170 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.778+0000 7f6b77a84640 1 -- 172.21.15.50:0/2280204674 >> 172.21.15.50:0/2280204674 conn(0x7f6b700cbbc0 msgr2=0x7f6b700cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.779+0000 7f6b77a84640 1 -- 172.21.15.50:0/2280204674 shutdown_connections 2023-10-19T10:37:22.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.779+0000 7f6b77a84640 1 -- 172.21.15.50:0/2280204674 wait complete. 2023-10-19T10:37:22.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.779+0000 7f0171ffb640 1 -- 172.21.15.50:0/2785442465 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164030a50 con 0x7f0174159040 2023-10-19T10:37:22.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.779+0000 7f0171ffb640 1 -- 172.21.15.50:0/2785442465 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164030bb0 con 0x7f0174159040 2023-10-19T10:37:22.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.779+0000 7f0171ffb640 1 -- 172.21.15.50:0/2785442465 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640383f0 con 0x7f0174159040 2023-10-19T10:37:22.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f14012770 con 0x7f2f1c069800 2023-10-19T10:37:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f0179711640 1 -- 172.21.15.50:0/2785442465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 msgr2=0x7f01740cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f0179711640 1 --2- 172.21.15.50:0/2785442465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 0x7f01740cd630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f016400de80 tx=0x7f016402feb0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f0179711640 1 -- 172.21.15.50:0/2785442465 shutdown_connections 2023-10-19T10:37:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f0179711640 1 --2- 172.21.15.50:0/2785442465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0174159040 0x7f01740cd630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f0179711640 1 -- 172.21.15.50:0/2785442465 >> 172.21.15.50:0/2785442465 conn(0x7f01740b0e50 msgr2=0x7f01740b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f2f127fc640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f0806a230 0x7f2f0806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f0179711640 1 -- 172.21.15.50:0/2785442465 shutdown_connections 2023-10-19T10:37:22.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.781+0000 7f2f216c9640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f0806a230 0x7f2f0806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.781+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2f140888e0 con 0x7f2f1c069800 2023-10-19T10:37:22.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.781+0000 7f2f216c9640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f0806a230 0x7f2f0806c6f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f18036000 tx=0x7f2f18008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.781+0000 7f0179711640 1 -- 172.21.15.50:0/2785442465 wait complete. 2023-10-19T10:37:22.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f0179711640 1 Processor -- start 2023-10-19T10:37:22.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7faf46404640 1 Processor -- start 2023-10-19T10:37:22.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f6b77a84640 1 Processor -- start 2023-10-19T10:37:22.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.780+0000 7f6b77a84640 1 -- start start 2023-10-19T10:37:22.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b77a84640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700c22e0 0x7f6b700c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b77a84640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 0x7f6b700c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b77a84640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b700ccd10 0x7f6b700ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b77a84640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70202160 con 0x7f6b700c9dc0 2023-10-19T10:37:22.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b77a84640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70202440 con 0x7f6b700ccd10 2023-10-19T10:37:22.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b77a84640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70202720 con 0x7f6b700c22e0 2023-10-19T10:37:22.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b7501f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 0x7f6b700c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b7501f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 0x7f6b700c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39134/0 (socket says 172.21.15.50:39134) 2023-10-19T10:37:22.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.782+0000 7f6b7501f640 1 -- 172.21.15.50:0/1744481468 learned_addr learned my addr 172.21.15.50:0/1744481468 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f6b7501f640 1 -- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700c22e0 msgr2=0x7f6b700c6210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-19T10:37:22.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f6b7501f640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700c22e0 0x7f6b700c6210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f6b7501f640 1 -- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b700ccd10 msgr2=0x7f6b700ca250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-19T10:37:22.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f6b7501f640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b700ccd10 0x7f6b700ca250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f6b7501f640 1 -- 172.21.15.50:0/1744481468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009400 con 0x7f6b700c9dc0 2023-10-19T10:37:22.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f6b7501f640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 0x7f6b700c6f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097a0 tx=0x7f6b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7faf46404640 1 -- start start 2023-10-19T10:37:22.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.783+0000 7f0179711640 1 -- start start 2023-10-19T10:37:22.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7faf46404640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faf4006ffe0 0x7faf40159460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7faf46404640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf400c9030 0x7faf400703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7f0179711640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01740ccf50 0x7f0174159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7faf46404640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faf400ceac0 0x7faf4015c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7faf46404640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4006b9c0 con 0x7faf4006ffe0 2023-10-19T10:37:22.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7faf46404640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf40070900 con 0x7faf400ceac0 2023-10-19T10:37:22.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7faf46404640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf400c97a0 con 0x7faf400c9030 2023-10-19T10:37:22.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.784+0000 7f0179711640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 0x7f01740c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f0179711640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 0x7f017414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f01727fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 0x7f01740c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f0172ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01740ccf50 0x7f0174159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f01727fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 0x7f01740c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39148/0 (socket says 172.21.15.50:39148) 2023-10-19T10:37:22.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f0172ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01740ccf50 0x7f0174159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43488/0 (socket says 172.21.15.50:43488) 2023-10-19T10:37:22.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f0179711640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740c6e40 con 0x7f0174159040 2023-10-19T10:37:22.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f01737fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 0x7f017414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f01737fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 0x7f017414f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:56102/0 (socket says 172.21.15.50:56102) 2023-10-19T10:37:22.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f01737fe640 1 -- 172.21.15.50:0/3625855702 learned_addr learned my addr 172.21.15.50:0/3625855702 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f0179711640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017415bc50 con 0x7f017415cd10 2023-10-19T10:37:22.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7f0179711640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017415bf30 con 0x7f01740ccf50 2023-10-19T10:37:22.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faf4006ffe0 0x7faf40159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faf4006ffe0 0x7faf40159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39144/0 (socket says 172.21.15.50:39144) 2023-10-19T10:37:22.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 -- 172.21.15.50:0/3146774512 learned_addr learned my addr 172.21.15.50:0/3146774512 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3f7fe640 1 --2- 172.21.15.50:0/3146774512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf400c9030 0x7faf400703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf449a1640 1 --2- 172.21.15.50:0/3146774512 >> v2:172.21.15.183:3300/0 conn(0x7faf400ceac0 0x7faf4015c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 -- 172.21.15.50:0/3146774512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf400c9030 msgr2=0x7faf400703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 --2- 172.21.15.50:0/3146774512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf400c9030 0x7faf400703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 -- 172.21.15.50:0/3146774512 >> v2:172.21.15.183:3300/0 conn(0x7faf400ceac0 msgr2=0x7faf4015c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 --2- 172.21.15.50:0/3146774512 >> v2:172.21.15.183:3300/0 conn(0x7faf400ceac0 0x7faf4015c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.785+0000 7faf3ffff640 1 -- 172.21.15.50:0/3146774512 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400c9a80 con 0x7faf4006ffe0 2023-10-19T10:37:22.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2f14053cc0 con 0x7f2f1c069800 2023-10-19T10:37:22.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60032cc0 con 0x7f6b700c9dc0 2023-10-19T10:37:22.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01737fe640 1 -- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01740ccf50 msgr2=0x7f0174159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b70202a00 con 0x7f6b700c9dc0 2023-10-19T10:37:22.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b70160730 con 0x7f6b700c9dc0 2023-10-19T10:37:22.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01737fe640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01740ccf50 0x7f0174159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01737fe640 1 -- 172.21.15.50:0/3625855702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 msgr2=0x7f01740c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.787+0000 7faf3ffff640 1 --2- 172.21.15.50:0/3146774512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf4006ffe0 0x7faf40159460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf4006f660 tx=0x7faf3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71301a0155380548 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01737fe640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 0x7f01740c3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01737fe640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016400db30 con 0x7f017415cd10 2023-10-19T10:37:22.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01727fc640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 0x7f01740c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.786+0000 7f01737fe640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 0x7f017414f330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f016800c340 tx=0x7f016800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.787+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680052d0 con 0x7f017415cd10 2023-10-19T10:37:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.787+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0168005450 con 0x7f017415cd10 2023-10-19T10:37:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.787+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168007260 con 0x7f017415cd10 2023-10-19T10:37:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.787+0000 7f0179711640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017415c210 con 0x7f017415cd10 2023-10-19T10:37:22.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.787+0000 7f0179711640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01740c3260 con 0x7f017415cd10 2023-10-19T10:37:22.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.788+0000 7faf3effd640 1 -- 172.21.15.50:0/3146774512 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30030860 con 0x7faf4006ffe0 2023-10-19T10:37:22.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.788+0000 7faf3effd640 1 -- 172.21.15.50:0/3146774512 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300309c0 con 0x7faf4006ffe0 2023-10-19T10:37:22.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.788+0000 7faf3effd640 1 -- 172.21.15.50:0/3146774512 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30030d50 con 0x7faf4006ffe0 2023-10-19T10:37:22.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.789+0000 7faf46404640 1 -- 172.21.15.50:0/3146774512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf4006ffe0 msgr2=0x7faf40159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.789+0000 7faf46404640 1 --2- 172.21.15.50:0/3146774512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf4006ffe0 0x7faf40159460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf4006f660 tx=0x7faf3002b9a0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.789+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60032e20 con 0x7f6b700c9dc0 2023-10-19T10:37:22.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.789+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600455f0 con 0x7f6b700c9dc0 2023-10-19T10:37:22.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.789+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6b6004d3f0 con 0x7f6b700c9dc0 2023-10-19T10:37:22.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.789+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0154005250 con 0x7f017415cd10 2023-10-19T10:37:22.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7f6b667fc640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b5406a0d0 0x7f6b5406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 -- 172.21.15.50:0/3146774512 shutdown_connections 2023-10-19T10:37:22.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 --2- 172.21.15.50:0/3146774512 >> v2:172.21.15.183:3300/0 conn(0x7faf400ceac0 0x7faf4015c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 --2- 172.21.15.50:0/3146774512 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf400c9030 0x7faf400703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 --2- 172.21.15.50:0/3146774512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf4006ffe0 0x7faf40159460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 -- 172.21.15.50:0/3146774512 >> 172.21.15.50:0/3146774512 conn(0x7faf400cdfd0 msgr2=0x7faf400ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6b600aa760 con 0x7f6b700c9dc0 2023-10-19T10:37:22.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 -- 172.21.15.50:0/3146774512 shutdown_connections 2023-10-19T10:37:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7f6b75820640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b5406a0d0 0x7f6b5406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b58005250 con 0x7f6b700c9dc0 2023-10-19T10:37:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.790+0000 7faf46404640 1 -- 172.21.15.50:0/3146774512 wait complete. 2023-10-19T10:37:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.792+0000 7faf46404640 1 Processor -- start 2023-10-19T10:37:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.792+0000 7faf46404640 1 -- start start 2023-10-19T10:37:22.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.793+0000 7faf46404640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 0x7faf400c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.793+0000 7faf46404640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf400c9030 0x7faf40164390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf46404640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf400ceac0 0x7faf4014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf46404640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401648d0 con 0x7faf400ceac0 2023-10-19T10:37:22.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf46404640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400c2dd0 con 0x7faf400c9030 2023-10-19T10:37:22.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf46404640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4014d190 con 0x7faf4006ffe0 2023-10-19T10:37:22.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 0x7faf400c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 0x7faf400c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43506/0 (socket says 172.21.15.50:43506) 2023-10-19T10:37:22.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 -- 172.21.15.50:0/3180952692 learned_addr learned my addr 172.21.15.50:0/3180952692 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf449a1640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf400ceac0 0x7faf4014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 -- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf400c9030 msgr2=0x7faf40164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf400c9030 0x7faf40164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 -- 172.21.15.50:0/3180952692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf400ceac0 msgr2=0x7faf4014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf400ceac0 0x7faf4014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.794+0000 7faf3ffff640 1 -- 172.21.15.50:0/3180952692 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300094d0 con 0x7faf4006ffe0 2023-10-19T10:37:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.792+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f01680073c0 con 0x7f017415cd10 2023-10-19T10:37:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.793+0000 7f015b7fe640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f016006a0a0 0x7f016006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.793+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0168001000 con 0x7f017415cd10 2023-10-19T10:37:22.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7faf3ffff640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 0x7faf400c40c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf30000c00 tx=0x7faf30004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7f0172ffd640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f016006a0a0 0x7f016006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30039a90 con 0x7faf4006ffe0 2023-10-19T10:37:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf400c3020 con 0x7faf4006ffe0 2023-10-19T10:37:22.836 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30039c10 con 0x7faf4006ffe0 2023-10-19T10:37:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf400c35e0 con 0x7faf4006ffe0 2023-10-19T10:37:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.795+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300433f0 con 0x7faf4006ffe0 2023-10-19T10:37:22.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.796+0000 7f0172ffd640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f016006a0a0 0x7f016006c560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0164039000 tx=0x7f0164034000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.796+0000 7f6b75820640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b5406a0d0 0x7f6b5406c590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b700c18b0 tx=0x7f6b6c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.797+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0168053710 con 0x7f017415cd10 2023-10-19T10:37:22.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.797+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4004e420 con 0x7faf4006ffe0 2023-10-19T10:37:22.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.798+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf3004c3b0 con 0x7faf4006ffe0 2023-10-19T10:37:22.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.799+0000 7faf3cff9640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf2c06a080 0x7faf2c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.799+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faf300aab80 con 0x7faf4006ffe0 2023-10-19T10:37:22.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.799+0000 7faf3f7fe640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf2c06a080 0x7faf2c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.800+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6b60075cf0 con 0x7f6b700c9dc0 2023-10-19T10:37:22.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.800+0000 7faf3f7fe640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf2c06a080 0x7faf2c06c540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf40169830 tx=0x7faf34009460 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.804+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7faf30075f90 con 0x7faf4006ffe0 2023-10-19T10:37:22.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.820+0000 7fa6c3f70640 1 Processor -- start 2023-10-19T10:37:22.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.820+0000 7fd99a01c640 1 Processor -- start 2023-10-19T10:37:22.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.820+0000 7fd99a01c640 1 -- start start 2023-10-19T10:37:22.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.821+0000 7fa6c3f70640 1 -- start start 2023-10-19T10:37:22.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.821+0000 7f0ef3d81640 1 Processor -- start 2023-10-19T10:37:22.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.821+0000 7fd99a01c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b4560 0x7fd9941619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.821+0000 7fd99a01c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940d0970 con 0x7fd99415aef0 2023-10-19T10:37:22.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.822+0000 7fd99a01c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99415b2d0 con 0x7fd9940b5200 2023-10-19T10:37:22.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.822+0000 7fd99a01c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940d0040 con 0x7fd9940b4560 2023-10-19T10:37:22.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.822+0000 7fd9937fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd99415aef0 0x7fd9940b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33614/0 (socket says 172.21.15.50:33614) 2023-10-19T10:37:22.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.822+0000 7fd9937fe640 1 -- 172.21.15.50:0/1857496813 learned_addr learned my addr 172.21.15.50:0/1857496813 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:22.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.821+0000 7f0ef3d81640 1 -- start start 2023-10-19T10:37:22.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.822+0000 7fd992ffd640 1 --2- 172.21.15.50:0/1857496813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b4560 0x7fd9941619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.822+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291346841 0 0) 0x7fd9940d0970 con 0x7fd99415aef0 2023-10-19T10:37:22.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c003520 con 0x7fd99415aef0 2023-10-19T10:37:22.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626243000 0 0) 0x7fd99415b2d0 con 0x7fd9940b5200 2023-10-19T10:37:22.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9940d0970 con 0x7fd9940b5200 2023-10-19T10:37:22.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef3d81640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0eec0c22e0 0x7f0eec071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef3d81640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0eec0cadd0 0x7f0eec0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef3d81640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec0cd6b0 con 0x7f0eec0ccd10 2023-10-19T10:37:22.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef3d81640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec0c26c0 con 0x7f0eec0cadd0 2023-10-19T10:37:22.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef3d81640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec06a380 con 0x7f0eec0c22e0 2023-10-19T10:37:22.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef131c640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0eec0ccd10 0x7f0eec06dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41956/0 (socket says 172.21.15.50:41956) 2023-10-19T10:37:22.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef131c640 1 -- 172.21.15.50:0/3643408596 learned_addr learned my addr 172.21.15.50:0/3643408596 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:22.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c3f70640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0b1fd0 0x7fa6bc070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c3f70640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa6bc070cc0 0x7fa6bc0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c3f70640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc0718a0 con 0x7fa6bc159550 2023-10-19T10:37:22.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c3f70640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6bc071150 con 0x7fa6bc070cc0 2023-10-19T10:37:22.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c3f70640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc06e610 con 0x7fa6bc0b1fd0 2023-10-19T10:37:22.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c1d0c640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fa6bc159550 0x7fa6bc0ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:41962/0 (socket says 172.21.15.50:41962) 2023-10-19T10:37:22.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c1d0c640 1 -- 172.21.15.50:0/1641867634 learned_addr learned my addr 172.21.15.50:0/1641867634 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:22.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fa6c150b640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0b1fd0 0x7fa6bc070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/3643408596 >> v2:172.21.15.183:3300/0 conn(0x7f0eec0c22e0 0x7f0eec071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7f0ef231e640 1 --2- 172.21.15.50:0/3643408596 >> v2:172.21.15.50:3300/0 conn(0x7f0eec0cadd0 0x7f0eec0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fd992ffd640 1 -- 172.21.15.50:0/1857496813 >> v1:172.21.15.183:6789/0 conn(0x7fd9940b5200 legacy=0x7fd9940b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fd992ffd640 1 -- 172.21.15.50:0/1857496813 >> v1:172.21.15.50:6789/0 conn(0x7fd99415aef0 legacy=0x7fd9940b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.823+0000 7fd992ffd640 1 -- 172.21.15.50:0/1857496813 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940d0320 con 0x7fd9940b4560 2023-10-19T10:37:22.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288010352 0 0) 0x7fa6bc0718a0 con 0x7fa6bc159550 2023-10-19T10:37:22.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a8003520 con 0x7fa6bc159550 2023-10-19T10:37:22.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c250d640 1 --2- 172.21.15.50:0/1641867634 >> v2:172.21.15.183:3300/0 conn(0x7fa6bc070cc0 0x7fa6bc0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7f0ef0b1b640 1 -- 172.21.15.50:0/3643408596 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203352668 0 0) 0x7f0eec0cd6b0 con 0x7f0eec0ccd10 2023-10-19T10:37:22.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7f0ef0b1b640 1 -- 172.21.15.50:0/3643408596 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed8003520 con 0x7f0eec0ccd10 2023-10-19T10:37:22.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7f0ef1b1d640 1 -- 172.21.15.50:0/3643408596 >> v2:172.21.15.50:3300/0 conn(0x7f0eec0cadd0 msgr2=0x7f0eec0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/3643408596 >> v2:172.21.15.50:3300/0 conn(0x7f0eec0cadd0 0x7f0eec0cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7f0ef1b1d640 1 -- 172.21.15.50:0/3643408596 >> v1:172.21.15.183:6789/0 conn(0x7f0eec0ccd10 legacy=0x7f0eec06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fd992ffd640 1 --2- 172.21.15.50:0/1857496813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b4560 0x7fd9941619d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9880097d0 tx=0x7fd98802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630dde40babeedfe server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98802d040 con 0x7fd9940b4560 2023-10-19T10:37:22.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd988031810 con 0x7fd9940b4560 2023-10-19T10:37:22.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fd9927fc640 1 -- 172.21.15.50:0/1857496813 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988031ba0 con 0x7fd9940b4560 2023-10-19T10:37:22.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166812320 0 0) 0x7fa6a8003520 con 0x7fa6bc159550 2023-10-19T10:37:22.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0b1fd0 msgr2=0x7fa6bc070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c0d0a640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0b1fd0 0x7fa6bc070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 >> v2:172.21.15.183:3300/0 conn(0x7fa6bc070cc0 msgr2=0x7fa6bc0cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 --2- 172.21.15.50:0/1641867634 >> v2:172.21.15.183:3300/0 conn(0x7fa6bc070cc0 0x7fa6bc0cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0ca8d0 con 0x7fa6bc159550 2023-10-19T10:37:22.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fa6b8001670 con 0x7fa6bc159550 2023-10-19T10:37:22.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 >> v1:172.21.15.183:6789/0 conn(0x7fa6bc159550 legacy=0x7fa6bc0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a8003970 0x7fa6a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a80047c0 0x7fa6a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a8005180 0x7fa6a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc071150 con 0x7fa6a8005180 2023-10-19T10:37:22.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc06e610 con 0x7fa6a8003970 2023-10-19T10:37:22.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800bf40 con 0x7fa6a80047c0 2023-10-19T10:37:22.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.824+0000 7f0ef1b1d640 1 -- 172.21.15.50:0/3643408596 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec160fa0 con 0x7f0eec0c22e0 2023-10-19T10:37:22.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fd99a01c640 1 -- 172.21.15.50:0/1857496813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b4560 msgr2=0x7fd9941619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fd99a01c640 1 --2- 172.21.15.50:0/1857496813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b4560 0x7fd9941619d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9880097d0 tx=0x7fd98802b8b0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7fa6c150b640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0b1fd0 0x7fa6bc070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:22.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c250d640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a8005180 0x7fa6a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.825+0000 7f0ef231e640 1 --2- 172.21.15.50:0/3643408596 >> v2:172.21.15.50:3300/0 conn(0x7f0eec0cadd0 0x7f0eec0cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:22.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c150b640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a80047c0 0x7fa6a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c1d0c640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a8003970 0x7fa6a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/3643408596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec071140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0663a0 tx=0x7f0ee802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5f95acc538dfe87 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c150b640 1 -- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a8003970 msgr2=0x7fa6a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c150b640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a8003970 0x7fa6a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c150b640 1 -- 172.21.15.50:0/1641867634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a8005180 msgr2=0x7fa6a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:22.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c150b640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a8005180 0x7fa6a800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.826+0000 7fa6c150b640 1 -- 172.21.15.50:0/1641867634 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8003520 con 0x7fa6a80047c0 2023-10-19T10:37:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7f0ef0b1b640 1 -- 172.21.15.50:0/3643408596 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8031640 con 0x7f0eec0c22e0 2023-10-19T10:37:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7f0ef0b1b640 1 -- 172.21.15.50:0/3643408596 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee80317a0 con 0x7f0eec0c22e0 2023-10-19T10:37:22.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7f0ef0b1b640 1 -- 172.21.15.50:0/3643408596 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8031b70 con 0x7f0eec0c22e0 2023-10-19T10:37:22.866 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7fd99a01c640 1 -- 172.21.15.50:0/1857496813 shutdown_connections 2023-10-19T10:37:22.866 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7fd99a01c640 1 --2- 172.21.15.50:0/1857496813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b4560 0x7fd9941619d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.866 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7fd99a01c640 1 -- 172.21.15.50:0/1857496813 >> 172.21.15.50:0/1857496813 conn(0x7fd9940cee50 msgr2=0x7fd9940cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7fd99a01c640 1 -- 172.21.15.50:0/1857496813 shutdown_connections 2023-10-19T10:37:22.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.827+0000 7fd99a01c640 1 -- 172.21.15.50:0/1857496813 wait complete. 2023-10-19T10:37:22.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7f0ef3d81640 1 -- 172.21.15.50:0/3643408596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 msgr2=0x7f0eec071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/3643408596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec071140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0663a0 tx=0x7f0ee802b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7f0ef3d81640 1 -- 172.21.15.50:0/3643408596 shutdown_connections 2023-10-19T10:37:22.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7fa6c150b640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a80047c0 0x7fa6a8004c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4010470 tx=0x7fa6a80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb3a4bfcf3c041f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/3643408596 >> v2:172.21.15.50:3300/0 conn(0x7f0eec0cadd0 0x7f0eec0cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/3643408596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec071140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.829+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400b0d0 con 0x7fa6a80047c0 2023-10-19T10:37:22.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.828+0000 7f0ef3d81640 1 -- 172.21.15.50:0/3643408596 >> 172.21.15.50:0/3643408596 conn(0x7f0eec0cbbc0 msgr2=0x7f0eec0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.829+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4011040 con 0x7fa6a80047c0 2023-10-19T10:37:22.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.829+0000 7fd99a01c640 1 Processor -- start 2023-10-19T10:37:22.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.829+0000 7fa6c0d0a640 1 -- 172.21.15.50:0/1641867634 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4003990 con 0x7fa6a80047c0 2023-10-19T10:37:22.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.829+0000 7f0ef3d81640 1 -- 172.21.15.50:0/3643408596 shutdown_connections 2023-10-19T10:37:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.830+0000 7fa6c3f70640 1 -- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a80047c0 msgr2=0x7fa6a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.830+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/1641867634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a80047c0 0x7fa6a8004c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4010470 tx=0x7fa6a80057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fa6c250d640 1 -- 172.21.15.50:0/1641867634 reap_dead start 2023-10-19T10:37:22.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fa6c3f70640 1 -- 172.21.15.50:0/1641867634 shutdown_connections 2023-10-19T10:37:22.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fa6c3f70640 1 -- 172.21.15.50:0/1641867634 >> 172.21.15.50:0/1641867634 conn(0x7fa6bc0cbbc0 msgr2=0x7fa6bc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:22.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.829+0000 7fd99a01c640 1 -- start start 2023-10-19T10:37:22.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.830+0000 7f0ef3d81640 1 -- 172.21.15.50:0/3643408596 wait complete. 2023-10-19T10:37:22.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fa6c3f70640 1 -- 172.21.15.50:0/1641867634 shutdown_connections 2023-10-19T10:37:22.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fa6c3f70640 1 -- 172.21.15.50:0/1641867634 wait complete. 2023-10-19T10:37:22.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fd99a01c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9940b4560 0x7fd9940c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fd99a01c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 0x7fd99414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7fd99a01c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 0x7fd9941626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd993fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 0x7fd9941626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd993fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 0x7fd9941626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39188/0 (socket says 172.21.15.50:39188) 2023-10-19T10:37:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd992ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 0x7fd99414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd992ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 0x7fd99414a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43536/0 (socket says 172.21.15.50:43536) 2023-10-19T10:37:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd993fff640 1 -- 172.21.15.50:0/2758738841 learned_addr learned my addr 172.21.15.50:0/2758738841 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cc6f0 con 0x7fd99415aef0 2023-10-19T10:37:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cbae0 con 0x7fd9940b4560 2023-10-19T10:37:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.832+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cbdc0 con 0x7fd9940b5200 2023-10-19T10:37:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fd993fff640 1 -- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 msgr2=0x7fd99414a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fd993fff640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 0x7fd99414a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fd993fff640 1 -- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9940b4560 msgr2=0x7fd9940c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fd993fff640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9940b4560 0x7fd9940c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fd993fff640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988009400 con 0x7fd99415aef0 2023-10-19T10:37:22.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fa6c3f70640 1 Processor -- start 2023-10-19T10:37:22.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7f0ef3d81640 1 Processor -- start 2023-10-19T10:37:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.831+0000 7f0ef3d81640 1 -- start start 2023-10-19T10:37:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef3d81640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef3d81640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 0x7f0eec0c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 0x7f0eec0c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 0x7f0eec0c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39192/0 (socket says 172.21.15.50:39192) 2023-10-19T10:37:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 -- 172.21.15.50:0/2198789338 learned_addr learned my addr 172.21.15.50:0/2198789338 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0eec0ccd10 0x7f0eec1ea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1f9390 con 0x7f0eec0cadd0 2023-10-19T10:37:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec164300 con 0x7f0eec0c22e0 2023-10-19T10:37:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1f95e0 con 0x7f0eec0ccd10 2023-10-19T10:37:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 -- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0eec0ccd10 msgr2=0x7f0eec1ea3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-19T10:37:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0eec0ccd10 0x7f0eec1ea3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 -- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 msgr2=0x7f0eec163dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7f0ef131c640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec163dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7f0ef131c640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc009400 con 0x7f0eec0cadd0 2023-10-19T10:37:22.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7f0ef131c640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 0x7f0eec0c8af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400cbc0 tx=0x7f0ee400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec163dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:22.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.833+0000 7fa6c3f70640 1 -- start start 2023-10-19T10:37:22.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fa6c3f70640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6bc0c4ef0 0x7fa6bc15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fa6c3f70640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 0x7fa6bc15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fa6c3f70640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6bc1562b0 0x7fa6bc156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fa6c3f70640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc156ca0 con 0x7fa6bc1562b0 2023-10-19T10:37:22.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fa6c3f70640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0ce1e0 con 0x7fa6bc0c4ef0 2023-10-19T10:37:22.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fa6c3f70640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0c7000 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fd993fff640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 0x7fd9941626d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00cac0 tx=0x7fd98c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fd992ffd640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 0x7fd99414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.834+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c003b70 con 0x7fd99415aef0 2023-10-19T10:37:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c003cd0 con 0x7fd99415aef0 2023-10-19T10:37:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00a960 con 0x7fd99415aef0 2023-10-19T10:37:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9940cc0a0 con 0x7fd99415aef0 2023-10-19T10:37:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99414af00 con 0x7fd99415aef0 2023-10-19T10:37:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 0x7fa6bc15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 0x7fa6bc15bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43546/0 (socket says 172.21.15.50:43546) 2023-10-19T10:37:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 -- 172.21.15.50:0/694390837 learned_addr learned my addr 172.21.15.50:0/694390837 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c1d0c640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6bc0c4ef0 0x7fa6bc15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 -- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6bc0c4ef0 msgr2=0x7fa6bc15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6bc0c4ef0 0x7fa6bc15b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 -- 172.21.15.50:0/694390837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6bc1562b0 msgr2=0x7fa6bc156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6bc1562b0 0x7fa6bc156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.835+0000 7fa6c150b640 1 -- 172.21.15.50:0/694390837 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc15c290 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.836+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400ad40 con 0x7f0eec0cadd0 2023-10-19T10:37:22.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.836+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee4012400 con 0x7f0eec0cadd0 2023-10-19T10:37:22.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.836+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8009400 con 0x7f0eec0cadd0 2023-10-19T10:37:22.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.836+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec1f9ba0 con 0x7f0eec0cadd0 2023-10-19T10:37:22.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.836+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400aea0 con 0x7f0eec0cadd0 2023-10-19T10:37:22.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.837+0000 7fa6c150b640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 0x7fa6bc15bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4010470 tx=0x7fa6b400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.837+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4004240 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.837+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40043a0 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.838+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4004710 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.838+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd98c00aac0 con 0x7fd99415aef0 2023-10-19T10:37:22.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.838+0000 7fd97bfff640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd98006a120 0x7fd98006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.839+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed4005250 con 0x7f0eec0cadd0 2023-10-19T10:37:22.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.839+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd974005250 con 0x7fd99415aef0 2023-10-19T10:37:22.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.839+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc1f9ad0 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.839+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc1fa090 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.839+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd98c088790 con 0x7fd99415aef0 2023-10-19T10:37:22.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.839+0000 7fd9937fe640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd98006a120 0x7fd98006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.840+0000 7fd9937fe640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd98006a120 0x7fd98006c5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9941633b0 tx=0x7fd984007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.840+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ee4007070 con 0x7f0eec0cadd0 2023-10-19T10:37:22.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.841+0000 7f0ee27fc640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ed006a120 0x7f0ed006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.841+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ee40887f0 con 0x7f0eec0cadd0 2023-10-19T10:37:22.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.841+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa6b403e420 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.841+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ed006a120 0x7f0ed006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.842+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc067260 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.842+0000 7fa6b27fc640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa6a006a150 0x7fa6a006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.842+0000 7f0ef1b1d640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ed006a120 0x7f0ed006c5e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8036000 tx=0x7f0ee803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.842+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa6b40aadb0 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.842+0000 7fa6c1d0c640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa6a006a150 0x7fa6a006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:22.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.843+0000 7fa6c1d0c640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa6a006a150 0x7fa6a006c610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8001050 tx=0x7fa6b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:22.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.845+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd98c057600 con 0x7fd99415aef0 2023-10-19T10:37:22.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.848+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa6b4076190 con 0x7fa6bc0c52d0 2023-10-19T10:37:22.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:22.853+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0ee4053d10 con 0x7f0eec0cadd0 2023-10-19T10:37:23.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.144+0000 7f40df999640 1 -- 172.21.15.50:0/921233357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f40d80c3f30 con 0x7f40d8070e90 2023-10-19T10:37:23.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.145+0000 7f40cdffb640 1 -- 172.21.15.50:0/921233357 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40c801e090 con 0x7f40d8070e90 2023-10-19T10:37:23.147 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2023-10-19T10:37:23.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.148+0000 7f40c37fe640 1 -- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f40bc06a1a0 msgr2=0x7f40bc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.148+0000 7f40c37fe640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f40bc06a1a0 0x7f40bc06c660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f40d81515b0 tx=0x7f40d4007ec0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.148+0000 7f40c37fe640 1 -- 172.21.15.50:0/921233357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 msgr2=0x7f40d8089560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.148+0000 7f40c37fe640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 0x7f40d8089560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40c800c7c0 tx=0x7f40c800cc80 comp rx=0 tx=0).stop 2023-10-19T10:37:23.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 -- 172.21.15.50:0/921233357 shutdown_connections 2023-10-19T10:37:23.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f40bc06a1a0 0x7f40bc06c660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d8159550 0x7f40d8153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80ccd10 0x7f40d815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 --2- 172.21.15.50:0/921233357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d8070e90 0x7f40d8089560 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 -- 172.21.15.50:0/921233357 >> 172.21.15.50:0/921233357 conn(0x7f40d80cbbc0 msgr2=0x7f40d8159930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.149+0000 7f40c37fe640 1 -- 172.21.15.50:0/921233357 shutdown_connections 2023-10-19T10:37:23.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.150+0000 7f40c37fe640 1 -- 172.21.15.50:0/921233357 wait complete. 2023-10-19T10:37:23.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-10-19T10:37:23.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.195+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2efc005500 con 0x7f2f1c069800 2023-10-19T10:37:23.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.196+0000 7f2f127fc640 1 -- 172.21.15.50:0/825486269 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f140572e0 con 0x7f2f1c069800 2023-10-19T10:37:23.199 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f0806a230 msgr2=0x7f2f0806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f0806a230 0x7f2f0806c6f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f18036000 tx=0x7f2f18008030 comp rx=0 tx=0).stop 2023-10-19T10:37:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 msgr2=0x7f2f1c14c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 0x7f2f1c14c720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400cda0 tx=0x7f2f1400a430 comp rx=0 tx=0).stop 2023-10-19T10:37:23.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 shutdown_connections 2023-10-19T10:37:23.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f0806a230 0x7f2f0806c6f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.199+0000 7f2f07fff640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f1c161b50 0x7f2f1c149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.200+0000 7f2f07fff640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1c0cc550 0x7f2f1c14cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.200+0000 7f2f07fff640 1 --2- 172.21.15.50:0/825486269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f1c069800 0x7f2f1c14c720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.200+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 >> 172.21.15.50:0/825486269 conn(0x7f2f1c0ce980 msgr2=0x7f2f1c0cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.200+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 shutdown_connections 2023-10-19T10:37:23.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.200+0000 7f2f07fff640 1 -- 172.21.15.50:0/825486269 wait complete. 2023-10-19T10:37:23.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.201+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f01540057d0 con 0x7f017415cd10 2023-10-19T10:37:23.205 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2023-10-19T10:37:23.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.202+0000 7f015b7fe640 1 -- 172.21.15.50:0/3625855702 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01680538b0 con 0x7f017415cd10 2023-10-19T10:37:23.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f016006a0a0 msgr2=0x7f016006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f016006a0a0 0x7f016006c560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0164039000 tx=0x7f0164034000 comp rx=0 tx=0).stop 2023-10-19T10:37:23.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 msgr2=0x7f017414f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 0x7f017414f330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f016800c340 tx=0x7f016800c800 comp rx=0 tx=0).stop 2023-10-19T10:37:23.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 shutdown_connections 2023-10-19T10:37:23.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f016006a0a0 0x7f016006c560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f017415cd10 0x7f017414f330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0174159040 0x7f01740c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 --2- 172.21.15.50:0/3625855702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f01740ccf50 0x7f0174159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.205+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 >> 172.21.15.50:0/3625855702 conn(0x7f01740b0e50 msgr2=0x7f01741e2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.206+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 shutdown_connections 2023-10-19T10:37:23.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.206+0000 7f01597fa640 1 -- 172.21.15.50:0/3625855702 wait complete. 2023-10-19T10:37:23.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.210+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6b58005500 con 0x7f6b700c9dc0 2023-10-19T10:37:23.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.211+0000 7f6b667fc640 1 -- 172.21.15.50:0/1744481468 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b60079310 con 0x7f6b700c9dc0 2023-10-19T10:37:23.214 INFO:teuthology.orchestra.run.smithi050.stdout:55834574859 2023-10-19T10:37:23.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.214+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b5406a0d0 msgr2=0x7f6b5406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.214+0000 7f6b77a84640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b5406a0d0 0x7f6b5406c590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b700c18b0 tx=0x7f6b6c0079b0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.214+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0ed4005500 con 0x7f0eec0cadd0 2023-10-19T10:37:23.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 msgr2=0x7f6b700c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 0x7f6b700c6f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097a0 tx=0x7f6b600046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 shutdown_connections 2023-10-19T10:37:23.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6b5406a0d0 0x7f6b5406c590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b700ccd10 0x7f6b700ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b700c9dc0 0x7f6b700c6f60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 --2- 172.21.15.50:0/1744481468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b700c22e0 0x7f6b700c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 >> 172.21.15.50:0/1744481468 conn(0x7f6b700cbbc0 msgr2=0x7f6b701e9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 shutdown_connections 2023-10-19T10:37:23.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f6b77a84640 1 -- 172.21.15.50:0/1744481468 wait complete. 2023-10-19T10:37:23.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.215+0000 7f0ee27fc640 1 -- 172.21.15.50:0/2198789338 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ee4057330 con 0x7f0eec0cadd0 2023-10-19T10:37:23.221 INFO:teuthology.orchestra.run.smithi050.stdout:55834574860 2023-10-19T10:37:23.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2023-10-19T10:37:23.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.217+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faf400b3f50 con 0x7faf4006ffe0 2023-10-19T10:37:23.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.217+0000 7faf3cff9640 1 -- 172.21.15.50:0/3180952692 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf300795b0 con 0x7faf4006ffe0 2023-10-19T10:37:23.223 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:23.224 INFO:teuthology.orchestra.run.smithi050.stdout:60129542155 2023-10-19T10:37:23.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ed006a120 msgr2=0x7f0ed006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ed006a120 0x7f0ed006c5e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8036000 tx=0x7f0ee803c040 comp rx=0 tx=0).stop 2023-10-19T10:37:23.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 msgr2=0x7f0eec0c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 0x7f0eec0c8af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400cbc0 tx=0x7f0ee400a430 comp rx=0 tx=0).stop 2023-10-19T10:37:23.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 shutdown_connections 2023-10-19T10:37:23.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ed006a120 0x7f0ed006c5e0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8036000 tx=0x7f0ee803c040 comp rx=0 tx=0).stop 2023-10-19T10:37:23.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0eec0ccd10 0x7f0eec1ea3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eec0cadd0 0x7f0eec0c8af0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 --2- 172.21.15.50:0/2198789338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eec0c22e0 0x7f0eec163dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 >> 172.21.15.50:0/2198789338 conn(0x7f0eec0cbbc0 msgr2=0x7f0eec0b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.218+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 shutdown_connections 2023-10-19T10:37:23.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.219+0000 7f0ef3d81640 1 -- 172.21.15.50:0/2198789338 wait complete. 2023-10-19T10:37:23.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf2c06a080 msgr2=0x7faf2c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf2c06a080 0x7faf2c06c540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf40169830 tx=0x7faf34009460 comp rx=0 tx=0).stop 2023-10-19T10:37:23.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 msgr2=0x7faf400c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 0x7faf400c40c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf30000c00 tx=0x7faf30004730 comp rx=0 tx=0).stop 2023-10-19T10:37:23.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 shutdown_connections 2023-10-19T10:37:23.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf2c06a080 0x7faf2c06c540 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf400ceac0 0x7faf4014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf400c9030 0x7faf40164390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 --2- 172.21.15.50:0/3180952692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf4006ffe0 0x7faf400c40c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 >> 172.21.15.50:0/3180952692 conn(0x7faf400cdfd0 msgr2=0x7faf40164cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 shutdown_connections 2023-10-19T10:37:23.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.220+0000 7faf46404640 1 -- 172.21.15.50:0/3180952692 wait complete. 2023-10-19T10:37:23.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2023-10-19T10:37:23.231 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:23.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.4 2023-10-19T10:37:23.233 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:23.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.0 2023-10-19T10:37:23.234 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:23.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.7 2023-10-19T10:37:23.236 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:23.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.252+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd974005500 con 0x7fd99415aef0 2023-10-19T10:37:23.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.252+0000 7fd97bfff640 1 -- 172.21.15.50:0/2758738841 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd98c057190 con 0x7fd99415aef0 2023-10-19T10:37:23.255 INFO:teuthology.orchestra.run.smithi050.stdout:55834574860 2023-10-19T10:37:23.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd98006a120 msgr2=0x7fd98006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd98006a120 0x7fd98006c5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9941633b0 tx=0x7fd984007930 comp rx=0 tx=0).stop 2023-10-19T10:37:23.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 msgr2=0x7fd9941626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 0x7fd9941626d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00cac0 tx=0x7fd98c00cf80 comp rx=0 tx=0).stop 2023-10-19T10:37:23.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 shutdown_connections 2023-10-19T10:37:23.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd98006a120 0x7fd98006c5e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd99415aef0 0x7fd9941626d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9940b5200 0x7fd99414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 --2- 172.21.15.50:0/2758738841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9940b4560 0x7fd9940c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 >> 172.21.15.50:0/2758738841 conn(0x7fd9940cee50 msgr2=0x7fd9940b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 shutdown_connections 2023-10-19T10:37:23.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.254+0000 7fd99a01c640 1 -- 172.21.15.50:0/2758738841 wait complete. 2023-10-19T10:37:23.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.258+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa6bc067260 con 0x7fa6bc0c52d0 2023-10-19T10:37:23.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.259+0000 7fa6b27fc640 1 -- 172.21.15.50:0/694390837 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6b40797b0 con 0x7fa6bc0c52d0 2023-10-19T10:37:23.261 INFO:teuthology.orchestra.run.smithi050.stdout:55834574859 2023-10-19T10:37:23.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa6a006a150 msgr2=0x7fa6a006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa6a006a150 0x7fa6a006c610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8001050 tx=0x7fa6b8009040 comp rx=0 tx=0).stop 2023-10-19T10:37:23.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 msgr2=0x7fa6bc15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:23.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 0x7fa6bc15bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4010470 tx=0x7fa6b400b960 comp rx=0 tx=0).stop 2023-10-19T10:37:23.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 shutdown_connections 2023-10-19T10:37:23.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa6a006a150 0x7fa6a006c610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6bc1562b0 0x7fa6bc156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6bc0c52d0 0x7fa6bc15bc30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 --2- 172.21.15.50:0/694390837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6bc0c4ef0 0x7fa6bc15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:23.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 >> 172.21.15.50:0/694390837 conn(0x7fa6bc0cbbc0 msgr2=0x7fa6bc0b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:23.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 shutdown_connections 2023-10-19T10:37:23.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:23.261+0000 7fa6c3f70640 1 -- 172.21.15.50:0/694390837 wait complete. 2023-10-19T10:37:23.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.2 2023-10-19T10:37:23.270 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:23.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2023-10-19T10:37:23.275 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:24.169 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-19T10:37:24.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.286+0000 7f5245349640 1 Processor -- start 2023-10-19T10:37:24.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.286+0000 7f5245349640 1 -- start start 2023-10-19T10:37:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f5245349640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f524015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f5245349640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524006aa30 con 0x7f524015c570 2023-10-19T10:37:24.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f5245349640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524006b2d0 con 0x7f52400cd920 2023-10-19T10:37:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f5245349640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524015ce90 con 0x7f52400d0e30 2023-10-19T10:37:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523effd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f524015c570 0x7f524006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33628/0 (socket says 172.21.15.50:33628) 2023-10-19T10:37:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523effd640 1 -- 172.21.15.50:0/2487855849 learned_addr learned my addr 172.21.15.50:0/2487855849 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f5237fff640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f524015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656181075 0 0) 0x7f524006aa30 con 0x7f524015c570 2023-10-19T10:37:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f522c003520 con 0x7f524015c570 2023-10-19T10:37:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252274620 0 0) 0x7f522c003520 con 0x7f524015c570 2023-10-19T10:37:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 msgr2=0x7f524015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f524015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 >> v1:172.21.15.183:6789/0 conn(0x7f52400cd920 legacy=0x7f52400d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400d1270 con 0x7f524015c570 2023-10-19T10:37:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f52300017f0 con 0x7f524015c570 2023-10-19T10:37:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 >> v1:172.21.15.50:6789/0 conn(0x7f524015c570 legacy=0x7f524006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f522c0039a0 0x7f522c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0047c0 0x7f522c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f522c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524015ce90 con 0x7f522c0039a0 2023-10-19T10:37:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c00bab0 con 0x7f522c0047c0 2023-10-19T10:37:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.287+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c00bc70 con 0x7f52400d0e30 2023-10-19T10:37:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523effd640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f522c0039a0 0x7f522c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f5237fff640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0047c0 0x7f522c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523f7fe640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f522c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523effd640 1 -- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 msgr2=0x7f522c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:24.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523effd640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f522c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523effd640 1 -- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0047c0 msgr2=0x7f522c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:24.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523effd640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0047c0 0x7f522c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523effd640 1 -- 172.21.15.50:0/2487855849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c003520 con 0x7f522c0039a0 2023-10-19T10:37:24.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.288+0000 7f523f7fe640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400d0e30 0x7f522c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:24.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f523effd640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f522c0039a0 0x7f522c003dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f523000c190 tx=0x7f522c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a688fb513c35d06d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230007270 con 0x7f522c0039a0 2023-10-19T10:37:24.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52300073f0 con 0x7f522c0039a0 2023-10-19T10:37:24.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f523e7fc640 1 -- 172.21.15.50:0/2487855849 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523002c8a0 con 0x7f522c0039a0 2023-10-19T10:37:24.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f5245349640 1 -- 172.21.15.50:0/2487855849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f522c0039a0 msgr2=0x7f522c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f5245349640 1 --2- 172.21.15.50:0/2487855849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f522c0039a0 0x7f522c003dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f523000c190 tx=0x7f522c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f523f7fe640 1 -- 172.21.15.50:0/2487855849 reap_dead start 2023-10-19T10:37:24.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f5245349640 1 -- 172.21.15.50:0/2487855849 shutdown_connections 2023-10-19T10:37:24.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f5245349640 1 -- 172.21.15.50:0/2487855849 >> 172.21.15.50:0/2487855849 conn(0x7f5240089ff0 msgr2=0x7f524009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:24.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f5245349640 1 -- 172.21.15.50:0/2487855849 shutdown_connections 2023-10-19T10:37:24.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.289+0000 7f5245349640 1 -- 172.21.15.50:0/2487855849 wait complete. 2023-10-19T10:37:24.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 Processor -- start 2023-10-19T10:37:24.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 -- start start 2023-10-19T10:37:24.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0039a0 0x7f524015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400c97e0 0x7f524015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 0x7f52400c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524015d990 con 0x7f524015d510 2023-10-19T10:37:24.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52400c6970 con 0x7f522c0039a0 2023-10-19T10:37:24.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.291+0000 7f5245349640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52400c6c50 con 0x7f52400c97e0 2023-10-19T10:37:24.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 0x7f52400c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 0x7f52400c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39224/0 (socket says 172.21.15.50:39224) 2023-10-19T10:37:24.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 -- 172.21.15.50:0/1819417500 learned_addr learned my addr 172.21.15.50:0/1819417500 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:24.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523effd640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0039a0 0x7f524015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f5237fff640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400c97e0 0x7f524015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 -- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400c97e0 msgr2=0x7f524015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400c97e0 0x7f524015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 -- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0039a0 msgr2=0x7f524015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0039a0 0x7f524015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 -- 172.21.15.50:0/1819417500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400c6f30 con 0x7f524015d510 2023-10-19T10:37:24.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f523f7fe640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 0x7f52400c7a10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5238000f80 tx=0x7f523800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238017070 con 0x7f524015d510 2023-10-19T10:37:24.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238003b80 con 0x7f524015d510 2023-10-19T10:37:24.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52400c7210 con 0x7f524015d510 2023-10-19T10:37:24.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238003d30 con 0x7f524015d510 2023-10-19T10:37:24.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.292+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524015fdd0 con 0x7f524015d510 2023-10-19T10:37:24.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.293+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5240073990 con 0x7f524015d510 2023-10-19T10:37:24.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.294+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52380050d0 con 0x7f524015d510 2023-10-19T10:37:24.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.294+0000 7f52377fe640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f522006a050 0x7f522006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.295+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5238087b00 con 0x7f524015d510 2023-10-19T10:37:24.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.295+0000 7f523effd640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f522006a050 0x7f522006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.295+0000 7f523effd640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f522006a050 0x7f522006c510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5230034000 tx=0x7f5230009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.299+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5238052f90 con 0x7f524015d510 2023-10-19T10:37:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.474+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f52400d3480 con 0x7f524015d510 2023-10-19T10:37:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.475+0000 7f52377fe640 1 -- 172.21.15.50:0/1819417500 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f52380565b0 con 0x7f524015d510 2023-10-19T10:37:24.478 INFO:teuthology.orchestra.run.smithi050.stdout:55834574859 2023-10-19T10:37:24.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f522006a050 msgr2=0x7f522006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f522006a050 0x7f522006c510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5230034000 tx=0x7f5230009030 comp rx=0 tx=0).stop 2023-10-19T10:37:24.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 msgr2=0x7f52400c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 0x7f52400c7a10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5238000f80 tx=0x7f523800ead0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 shutdown_connections 2023-10-19T10:37:24.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f522006a050 0x7f522006c510 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f524015d510 0x7f52400c7a10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52400c97e0 0x7f524015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 --2- 172.21.15.50:0/1819417500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f522c0039a0 0x7f524015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 >> 172.21.15.50:0/1819417500 conn(0x7f5240089ff0 msgr2=0x7f52400ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:24.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 shutdown_connections 2023-10-19T10:37:24.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.477+0000 7f5245349640 1 -- 172.21.15.50:0/1819417500 wait complete. 2023-10-19T10:37:24.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2023-10-19T10:37:24.490 DEBUG:teuthology.parallel:result is None 2023-10-19T10:37:24.491 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-19T10:37:24.491 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:37:24.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.594+0000 7fe74f041640 1 Processor -- start 2023-10-19T10:37:24.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.594+0000 7fe74f041640 1 -- start start 2023-10-19T10:37:24.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74f041640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe7480d2db0 0x7fe74815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74f041640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74806aa30 con 0x7fe7480d0e30 2023-10-19T10:37:24.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74f041640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74806b2d0 con 0x7fe74815c570 2023-10-19T10:37:24.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74f041640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74815ce90 con 0x7fe7480d2db0 2023-10-19T10:37:24.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe747fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe7480d0e30 0x7fe7480d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33634/0 (socket says 172.21.15.50:33634) 2023-10-19T10:37:24.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe747fff640 1 -- 172.21.15.50:0/514356747 learned_addr learned my addr 172.21.15.50:0/514356747 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:24.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74d5de640 1 --2- 172.21.15.50:0/514356747 >> v2:172.21.15.50:3300/0 conn(0x7fe7480d2db0 0x7fe74815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74d5de640 1 -- 172.21.15.50:0/514356747 >> v1:172.21.15.183:6789/0 conn(0x7fe74815c570 legacy=0x7fe74806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:24.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.595+0000 7fe74d5de640 1 -- 172.21.15.50:0/514356747 >> v1:172.21.15.50:6789/0 conn(0x7fe7480d0e30 legacy=0x7fe7480d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.596+0000 7fe7477fe640 1 -- 172.21.15.50:0/514356747 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724584348 0 0) 0x7fe74806aa30 con 0x7fe7480d0e30 2023-10-19T10:37:24.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.596+0000 7fe74d5de640 1 -- 172.21.15.50:0/514356747 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480d1270 con 0x7fe7480d2db0 2023-10-19T10:37:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.596+0000 7fe74d5de640 1 --2- 172.21.15.50:0/514356747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe74815c950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7300097a0 tx=0x7fe73002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb8399546bae620 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.596+0000 7fe7477fe640 1 -- 172.21.15.50:0/514356747 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73002e070 con 0x7fe7480d2db0 2023-10-19T10:37:24.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.596+0000 7fe7477fe640 1 -- 172.21.15.50:0/514356747 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7300315a0 con 0x7fe7480d2db0 2023-10-19T10:37:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.596+0000 7fe7477fe640 1 -- 172.21.15.50:0/514356747 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730031930 con 0x7fe7480d2db0 2023-10-19T10:37:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 -- 172.21.15.50:0/514356747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 msgr2=0x7fe74815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 --2- 172.21.15.50:0/514356747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe74815c950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7300097a0 tx=0x7fe73002b840 comp rx=0 tx=0).stop 2023-10-19T10:37:24.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 -- 172.21.15.50:0/514356747 shutdown_connections 2023-10-19T10:37:24.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 --2- 172.21.15.50:0/514356747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe74815c950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 -- 172.21.15.50:0/514356747 >> 172.21.15.50:0/514356747 conn(0x7fe748089ff0 msgr2=0x7fe74809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:24.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 -- 172.21.15.50:0/514356747 shutdown_connections 2023-10-19T10:37:24.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.597+0000 7fe74f041640 1 -- 172.21.15.50:0/514356747 wait complete. 2023-10-19T10:37:24.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.598+0000 7fe74f041640 1 Processor -- start 2023-10-19T10:37:24.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.598+0000 7fe74f041640 1 -- start start 2023-10-19T10:37:24.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74f041640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7480d0e30 0x7fe74815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74f041640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe7481f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74f041640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe74815c570 0x7fe74815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74f041640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815fea0 con 0x7fe7480d2db0 2023-10-19T10:37:24.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74f041640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815b0c0 con 0x7fe74815c570 2023-10-19T10:37:24.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74f041640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815ef20 con 0x7fe7480d0e30 2023-10-19T10:37:24.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe747fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe7481f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe747fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe7481f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39256/0 (socket says 172.21.15.50:39256) 2023-10-19T10:37:24.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe747fff640 1 -- 172.21.15.50:0/3832225094 learned_addr learned my addr 172.21.15.50:0/3832225094 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:24.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74d5de640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe74815c570 0x7fe74815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.599+0000 7fe74cddd640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7480d0e30 0x7fe74815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe747fff640 1 -- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7480d0e30 msgr2=0x7fe74815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe747fff640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7480d0e30 0x7fe74815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe747fff640 1 -- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe74815c570 msgr2=0x7fe74815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe747fff640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe74815c570 0x7fe74815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe747fff640 1 -- 172.21.15.50:0/3832225094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730009400 con 0x7fe7480d2db0 2023-10-19T10:37:24.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe747fff640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe7481f2590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe73800c430 tx=0x7fe73800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7380074e0 con 0x7fe7480d2db0 2023-10-19T10:37:24.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe738007660 con 0x7fe7480d2db0 2023-10-19T10:37:24.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74815f8e0 con 0x7fe7480d2db0 2023-10-19T10:37:24.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738004a20 con 0x7fe7480d2db0 2023-10-19T10:37:24.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.600+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74815f1c0 con 0x7fe7480d2db0 2023-10-19T10:37:24.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.601+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74815ed80 con 0x7fe7480d2db0 2023-10-19T10:37:24.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.601+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe738004b80 con 0x7fe7480d2db0 2023-10-19T10:37:24.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.602+0000 7fe7457fa640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe72806a050 0x7fe72806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.602+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe7380886d0 con 0x7fe7480d2db0 2023-10-19T10:37:24.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.602+0000 7fe74cddd640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe72806a050 0x7fe72806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.602+0000 7fe74cddd640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe72806a050 0x7fe72806c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe740005550 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.606+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe74815ed80 con 0x7fe7480d2db0 2023-10-19T10:37:24.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.777+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe74806ac80 con 0x7fe72806a050 2023-10-19T10:37:24.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.787+0000 7fe7457fa640 1 -- 172.21.15.50:0/3832225094 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367360 (secure 0 0 0) 0x7fe7400051f0 con 0x7fe72806a050 2023-10-19T10:37:24.793 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:24.795 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:37:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe72806a050 msgr2=0x7fe72806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe72806a050 0x7fe72806c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe740005550 tx=0x7fe740009040 comp rx=0 tx=0).stop 2023-10-19T10:37:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 msgr2=0x7fe7481f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe7481f2590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe73800c430 tx=0x7fe73800c8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 shutdown_connections 2023-10-19T10:37:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe72806a050 0x7fe72806c510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe74815c570 0x7fe74815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7480d2db0 0x7fe7481f2590 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 --2- 172.21.15.50:0/3832225094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7480d0e30 0x7fe74815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 >> 172.21.15.50:0/3832225094 conn(0x7fe748089ff0 msgr2=0x7fe7480b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:24.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 shutdown_connections 2023-10-19T10:37:24.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.790+0000 7fe74f041640 1 -- 172.21.15.50:0/3832225094 wait complete. 2023-10-19T10:37:24.804 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-10-19T10:37:24.470680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236080,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738640,"statfs":{"total":773094113280,"available":772852367360,"internally_reserved":0,"allocated":3579904,"data_stored":2146982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003722"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096693+0000","last_change":"2023-10-19T10:36:59.359352+0000","last_active":"2023-10-19T10:37:00.096693+0000","last_peered":"2023-10-19T10:37:00.096693+0000","last_clean":"2023-10-19T10:37:00.096693+0000","last_became_active":"2023-10-19T10:36:59.358992+0000","last_became_peered":"2023-10-19T10:36:59.358992+0000","last_unstale":"2023-10-19T10:37:00.096693+0000","last_undegraded":"2023-10-19T10:37:00.096693+0000","last_fullsized":"2023-10-19T10:37:00.096693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:36:39.869930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278383+0000","last_change":"2023-10-19T10:36:54.305740+0000","last_active":"2023-10-19T10:37:00.278383+0000","last_peered":"2023-10-19T10:37:00.278383+0000","last_clean":"2023-10-19T10:37:00.278383+0000","last_became_active":"2023-10-19T10:36:54.305192+0000","last_became_peered":"2023-10-19T10:36:54.305192+0000","last_unstale":"2023-10-19T10:37:00.278383+0000","last_undegraded":"2023-10-19T10:37:00.278383+0000","last_fullsized":"2023-10-19T10:37:00.278383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097666+0000","last_change":"2023-10-19T10:36:55.108193+0000","last_active":"2023-10-19T10:37:00.097666+0000","last_peered":"2023-10-19T10:37:00.097666+0000","last_clean":"2023-10-19T10:37:00.097666+0000","last_became_active":"2023-10-19T10:36:55.107375+0000","last_became_peered":"2023-10-19T10:36:55.107375+0000","last_unstale":"2023-10-19T10:37:00.097666+0000","last_undegraded":"2023-10-19T10:37:00.097666+0000","last_fullsized":"2023-10-19T10:37:00.097666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:13:03.441540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097567+0000","last_change":"2023-10-19T10:36:59.344962+0000","last_active":"2023-10-19T10:37:00.097567+0000","last_peered":"2023-10-19T10:37:00.097567+0000","last_clean":"2023-10-19T10:37:00.097567+0000","last_became_active":"2023-10-19T10:36:59.344519+0000","last_became_peered":"2023-10-19T10:36:59.344519+0000","last_unstale":"2023-10-19T10:37:00.097567+0000","last_undegraded":"2023-10-19T10:37:00.097567+0000","last_fullsized":"2023-10-19T10:37:00.097567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:33.660694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:54.310896+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:54.310583+0000","last_became_peered":"2023-10-19T10:36:54.310583+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:56:34.283482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323668+0000","last_change":"2023-10-19T10:36:55.104115+0000","last_active":"2023-10-19T10:37:00.323668+0000","last_peered":"2023-10-19T10:37:00.323668+0000","last_clean":"2023-10-19T10:37:00.323668+0000","last_became_active":"2023-10-19T10:36:55.103785+0000","last_became_peered":"2023-10-19T10:36:55.103785+0000","last_unstale":"2023-10-19T10:37:00.323668+0000","last_undegraded":"2023-10-19T10:37:00.323668+0000","last_fullsized":"2023-10-19T10:37:00.323668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324072+0000","last_change":"2023-10-19T10:36:59.350493+0000","last_active":"2023-10-19T10:37:00.324072+0000","last_peered":"2023-10-19T10:37:00.324072+0000","last_clean":"2023-10-19T10:37:00.324072+0000","last_became_active":"2023-10-19T10:36:59.350100+0000","last_became_peered":"2023-10-19T10:36:59.350100+0000","last_unstale":"2023-10-19T10:37:00.324072+0000","last_undegraded":"2023-10-19T10:37:00.324072+0000","last_fullsized":"2023-10-19T10:37:00.324072+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324355+0000","last_change":"2023-10-19T10:36:54.307087+0000","last_active":"2023-10-19T10:37:00.324355+0000","last_peered":"2023-10-19T10:37:00.324355+0000","last_clean":"2023-10-19T10:37:00.324355+0000","last_became_active":"2023-10-19T10:36:54.306818+0000","last_became_peered":"2023-10-19T10:36:54.306818+0000","last_unstale":"2023-10-19T10:37:00.324355+0000","last_undegraded":"2023-10-19T10:37:00.324355+0000","last_fullsized":"2023-10-19T10:37:00.324355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:25.504656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098032+0000","last_change":"2023-10-19T10:36:55.111676+0000","last_active":"2023-10-19T10:37:00.098032+0000","last_peered":"2023-10-19T10:37:00.098032+0000","last_clean":"2023-10-19T10:37:00.098032+0000","last_became_active":"2023-10-19T10:36:55.110442+0000","last_became_peered":"2023-10-19T10:36:55.110442+0000","last_unstale":"2023-10-19T10:37:00.098032+0000","last_undegraded":"2023-10-19T10:37:00.098032+0000","last_fullsized":"2023-10-19T10:37:00.098032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:51:02.897898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097851+0000","last_change":"2023-10-19T10:36:59.357016+0000","last_active":"2023-10-19T10:37:00.097851+0000","last_peered":"2023-10-19T10:37:00.097851+0000","last_clean":"2023-10-19T10:37:00.097851+0000","last_became_active":"2023-10-19T10:36:59.356666+0000","last_became_peered":"2023-10-19T10:36:59.356666+0000","last_unstale":"2023-10-19T10:37:00.097851+0000","last_undegraded":"2023-10-19T10:37:00.097851+0000","last_fullsized":"2023-10-19T10:37:00.097851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324054+0000","last_change":"2023-10-19T10:36:54.306526+0000","last_active":"2023-10-19T10:37:00.324054+0000","last_peered":"2023-10-19T10:37:00.324054+0000","last_clean":"2023-10-19T10:37:00.324054+0000","last_became_active":"2023-10-19T10:36:54.306285+0000","last_became_peered":"2023-10-19T10:36:54.306285+0000","last_unstale":"2023-10-19T10:37:00.324054+0000","last_undegraded":"2023-10-19T10:37:00.324054+0000","last_fullsized":"2023-10-19T10:37:00.324054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:55.717306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098946+0000","last_change":"2023-10-19T10:36:55.115029+0000","last_active":"2023-10-19T10:37:00.098946+0000","last_peered":"2023-10-19T10:37:00.098946+0000","last_clean":"2023-10-19T10:37:00.098946+0000","last_became_active":"2023-10-19T10:36:55.114754+0000","last_became_peered":"2023-10-19T10:36:55.114754+0000","last_unstale":"2023-10-19T10:37:00.098946+0000","last_undegraded":"2023-10-19T10:37:00.098946+0000","last_fullsized":"2023-10-19T10:37:00.098946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:30:31.789571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097648+0000","last_change":"2023-10-19T10:36:59.355687+0000","last_active":"2023-10-19T10:37:00.097648+0000","last_peered":"2023-10-19T10:37:00.097648+0000","last_clean":"2023-10-19T10:37:00.097648+0000","last_became_active":"2023-10-19T10:36:59.355294+0000","last_became_peered":"2023-10-19T10:36:59.355294+0000","last_unstale":"2023-10-19T10:37:00.097648+0000","last_undegraded":"2023-10-19T10:37:00.097648+0000","last_fullsized":"2023-10-19T10:37:00.097648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:39:00.759461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278352+0000","last_change":"2023-10-19T10:36:54.307829+0000","last_active":"2023-10-19T10:37:00.278352+0000","last_peered":"2023-10-19T10:37:00.278352+0000","last_clean":"2023-10-19T10:37:00.278352+0000","last_became_active":"2023-10-19T10:36:54.307571+0000","last_became_peered":"2023-10-19T10:36:54.307571+0000","last_unstale":"2023-10-19T10:37:00.278352+0000","last_undegraded":"2023-10-19T10:37:00.278352+0000","last_fullsized":"2023-10-19T10:37:00.278352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:45.887676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323888+0000","last_change":"2023-10-19T10:36:55.112212+0000","last_active":"2023-10-19T10:37:00.323888+0000","last_peered":"2023-10-19T10:37:00.323888+0000","last_clean":"2023-10-19T10:37:00.323888+0000","last_became_active":"2023-10-19T10:36:55.111854+0000","last_became_peered":"2023-10-19T10:36:55.111854+0000","last_unstale":"2023-10-19T10:37:00.323888+0000","last_undegraded":"2023-10-19T10:37:00.323888+0000","last_fullsized":"2023-10-19T10:37:00.323888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:49:46.713742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099140+0000","last_change":"2023-10-19T10:36:59.352206+0000","last_active":"2023-10-19T10:37:00.099140+0000","last_peered":"2023-10-19T10:37:00.099140+0000","last_clean":"2023-10-19T10:37:00.099140+0000","last_became_active":"2023-10-19T10:36:59.351813+0000","last_became_peered":"2023-10-19T10:36:59.351813+0000","last_unstale":"2023-10-19T10:37:00.099140+0000","last_undegraded":"2023-10-19T10:37:00.099140+0000","last_fullsized":"2023-10-19T10:37:00.099140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:02.738858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098273+0000","last_change":"2023-10-19T10:36:54.305390+0000","last_active":"2023-10-19T10:37:00.098273+0000","last_peered":"2023-10-19T10:37:00.098273+0000","last_clean":"2023-10-19T10:37:00.098273+0000","last_became_active":"2023-10-19T10:36:54.304960+0000","last_became_peered":"2023-10-19T10:36:54.304960+0000","last_unstale":"2023-10-19T10:37:00.098273+0000","last_undegraded":"2023-10-19T10:37:00.098273+0000","last_fullsized":"2023-10-19T10:37:00.098273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:21.713184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324108+0000","last_change":"2023-10-19T10:36:55.114266+0000","last_active":"2023-10-19T10:37:00.324108+0000","last_peered":"2023-10-19T10:37:00.324108+0000","last_clean":"2023-10-19T10:37:00.324108+0000","last_became_active":"2023-10-19T10:36:55.113992+0000","last_became_peered":"2023-10-19T10:36:55.113992+0000","last_unstale":"2023-10-19T10:37:00.324108+0000","last_undegraded":"2023-10-19T10:37:00.324108+0000","last_fullsized":"2023-10-19T10:37:00.324108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:17.080140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323688+0000","last_change":"2023-10-19T10:36:59.349278+0000","last_active":"2023-10-19T10:37:00.323688+0000","last_peered":"2023-10-19T10:37:00.323688+0000","last_clean":"2023-10-19T10:37:00.323688+0000","last_became_active":"2023-10-19T10:36:59.348852+0000","last_became_peered":"2023-10-19T10:36:59.348852+0000","last_unstale":"2023-10-19T10:37:00.323688+0000","last_undegraded":"2023-10-19T10:37:00.323688+0000","last_fullsized":"2023-10-19T10:37:00.323688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:20.365692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096958+0000","last_change":"2023-10-19T10:36:54.306039+0000","last_active":"2023-10-19T10:37:00.096958+0000","last_peered":"2023-10-19T10:37:00.096958+0000","last_clean":"2023-10-19T10:37:00.096958+0000","last_became_active":"2023-10-19T10:36:54.304502+0000","last_became_peered":"2023-10-19T10:36:54.304502+0000","last_unstale":"2023-10-19T10:37:00.096958+0000","last_undegraded":"2023-10-19T10:37:00.096958+0000","last_fullsized":"2023-10-19T10:37:00.096958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:07:52.900648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278255+0000","last_change":"2023-10-19T10:36:55.114034+0000","last_active":"2023-10-19T10:37:00.278255+0000","last_peered":"2023-10-19T10:37:00.278255+0000","last_clean":"2023-10-19T10:37:00.278255+0000","last_became_active":"2023-10-19T10:36:55.113734+0000","last_became_peered":"2023-10-19T10:36:55.113734+0000","last_unstale":"2023-10-19T10:37:00.278255+0000","last_undegraded":"2023-10-19T10:37:00.278255+0000","last_fullsized":"2023-10-19T10:37:00.278255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278113+0000","last_change":"2023-10-19T10:36:59.349586+0000","last_active":"2023-10-19T10:37:00.278113+0000","last_peered":"2023-10-19T10:37:00.278113+0000","last_clean":"2023-10-19T10:37:00.278113+0000","last_became_active":"2023-10-19T10:36:59.349176+0000","last_became_peered":"2023-10-19T10:36:59.349176+0000","last_unstale":"2023-10-19T10:37:00.278113+0000","last_undegraded":"2023-10-19T10:37:00.278113+0000","last_fullsized":"2023-10-19T10:37:00.278113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323262+0000","last_change":"2023-10-19T10:36:54.310735+0000","last_active":"2023-10-19T10:37:00.323262+0000","last_peered":"2023-10-19T10:37:00.323262+0000","last_clean":"2023-10-19T10:37:00.323262+0000","last_became_active":"2023-10-19T10:36:54.309734+0000","last_became_peered":"2023-10-19T10:36:54.309734+0000","last_unstale":"2023-10-19T10:37:00.323262+0000","last_undegraded":"2023-10-19T10:37:00.323262+0000","last_fullsized":"2023-10-19T10:37:00.323262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323712+0000","last_change":"2023-10-19T10:36:55.113078+0000","last_active":"2023-10-19T10:37:00.323712+0000","last_peered":"2023-10-19T10:37:00.323712+0000","last_clean":"2023-10-19T10:37:00.323712+0000","last_became_active":"2023-10-19T10:36:55.112775+0000","last_became_peered":"2023-10-19T10:36:55.112775+0000","last_unstale":"2023-10-19T10:37:00.323712+0000","last_undegraded":"2023-10-19T10:37:00.323712+0000","last_fullsized":"2023-10-19T10:37:00.323712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:12:41.211214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322933+0000","last_change":"2023-10-19T10:36:55.105391+0000","last_active":"2023-10-19T10:37:00.322933+0000","last_peered":"2023-10-19T10:37:00.322933+0000","last_clean":"2023-10-19T10:37:00.322933+0000","last_became_active":"2023-10-19T10:36:55.105112+0000","last_became_peered":"2023-10-19T10:36:55.105112+0000","last_unstale":"2023-10-19T10:37:00.322933+0000","last_undegraded":"2023-10-19T10:37:00.322933+0000","last_fullsized":"2023-10-19T10:37:00.322933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278476+0000","last_change":"2023-10-19T10:36:54.305435+0000","last_active":"2023-10-19T10:37:00.278476+0000","last_peered":"2023-10-19T10:37:00.278476+0000","last_clean":"2023-10-19T10:37:00.278476+0000","last_became_active":"2023-10-19T10:36:54.305123+0000","last_became_peered":"2023-10-19T10:36:54.305123+0000","last_unstale":"2023-10-19T10:37:00.278476+0000","last_undegraded":"2023-10-19T10:37:00.278476+0000","last_fullsized":"2023-10-19T10:37:00.278476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323209+0000","last_change":"2023-10-19T10:36:59.350259+0000","last_active":"2023-10-19T10:37:00.323209+0000","last_peered":"2023-10-19T10:37:00.323209+0000","last_clean":"2023-10-19T10:37:00.323209+0000","last_became_active":"2023-10-19T10:36:59.349878+0000","last_became_peered":"2023-10-19T10:36:59.349878+0000","last_unstale":"2023-10-19T10:37:00.323209+0000","last_undegraded":"2023-10-19T10:37:00.323209+0000","last_fullsized":"2023-10-19T10:37:00.323209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323250+0000","last_change":"2023-10-19T10:36:55.113112+0000","last_active":"2023-10-19T10:37:00.323250+0000","last_peered":"2023-10-19T10:37:00.323250+0000","last_clean":"2023-10-19T10:37:00.323250+0000","last_became_active":"2023-10-19T10:36:55.110232+0000","last_became_peered":"2023-10-19T10:36:55.110232+0000","last_unstale":"2023-10-19T10:37:00.323250+0000","last_undegraded":"2023-10-19T10:37:00.323250+0000","last_fullsized":"2023-10-19T10:37:00.323250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324722+0000","last_change":"2023-10-19T10:36:54.310064+0000","last_active":"2023-10-19T10:37:00.324722+0000","last_peered":"2023-10-19T10:37:00.324722+0000","last_clean":"2023-10-19T10:37:00.324722+0000","last_became_active":"2023-10-19T10:36:54.309581+0000","last_became_peered":"2023-10-19T10:36:54.309581+0000","last_unstale":"2023-10-19T10:37:00.324722+0000","last_undegraded":"2023-10-19T10:37:00.324722+0000","last_fullsized":"2023-10-19T10:37:00.324722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:16.076243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098023+0000","last_change":"2023-10-19T10:36:59.355292+0000","last_active":"2023-10-19T10:37:00.098023+0000","last_peered":"2023-10-19T10:37:00.098023+0000","last_clean":"2023-10-19T10:37:00.098023+0000","last_became_active":"2023-10-19T10:36:59.354866+0000","last_became_peered":"2023-10-19T10:36:59.354866+0000","last_unstale":"2023-10-19T10:37:00.098023+0000","last_undegraded":"2023-10-19T10:37:00.098023+0000","last_fullsized":"2023-10-19T10:37:00.098023+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098814+0000","last_change":"2023-10-19T10:36:54.310722+0000","last_active":"2023-10-19T10:37:00.098814+0000","last_peered":"2023-10-19T10:37:00.098814+0000","last_clean":"2023-10-19T10:37:00.098814+0000","last_became_active":"2023-10-19T10:36:54.310410+0000","last_became_peered":"2023-10-19T10:36:54.310410+0000","last_unstale":"2023-10-19T10:37:00.098814+0000","last_undegraded":"2023-10-19T10:37:00.098814+0000","last_fullsized":"2023-10-19T10:37:00.098814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:00.221492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097163+0000","last_change":"2023-10-19T10:36:55.110762+0000","last_active":"2023-10-19T10:37:00.097163+0000","last_peered":"2023-10-19T10:37:00.097163+0000","last_clean":"2023-10-19T10:37:00.097163+0000","last_became_active":"2023-10-19T10:36:55.110503+0000","last_became_peered":"2023-10-19T10:36:55.110503+0000","last_unstale":"2023-10-19T10:37:00.097163+0000","last_undegraded":"2023-10-19T10:37:00.097163+0000","last_fullsized":"2023-10-19T10:37:00.097163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:14.103808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323380+0000","last_change":"2023-10-19T10:36:59.352779+0000","last_active":"2023-10-19T10:37:00.323380+0000","last_peered":"2023-10-19T10:37:00.323380+0000","last_clean":"2023-10-19T10:37:00.323380+0000","last_became_active":"2023-10-19T10:36:59.352362+0000","last_became_peered":"2023-10-19T10:36:59.352362+0000","last_unstale":"2023-10-19T10:37:00.323380+0000","last_undegraded":"2023-10-19T10:37:00.323380+0000","last_fullsized":"2023-10-19T10:37:00.323380+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:53.134651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:04.746420+0000","last_change":"2023-10-19T10:36:54.310368+0000","last_active":"2023-10-19T10:37:04.746420+0000","last_peered":"2023-10-19T10:37:04.746420+0000","last_clean":"2023-10-19T10:37:04.746420+0000","last_became_active":"2023-10-19T10:36:54.309745+0000","last_became_peered":"2023-10-19T10:36:54.309745+0000","last_unstale":"2023-10-19T10:37:04.746420+0000","last_undegraded":"2023-10-19T10:37:04.746420+0000","last_fullsized":"2023-10-19T10:37:04.746420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:15:27.898150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099416+0000","last_change":"2023-10-19T10:36:55.106300+0000","last_active":"2023-10-19T10:37:00.099416+0000","last_peered":"2023-10-19T10:37:00.099416+0000","last_clean":"2023-10-19T10:37:00.099416+0000","last_became_active":"2023-10-19T10:36:55.106045+0000","last_became_peered":"2023-10-19T10:36:55.106045+0000","last_unstale":"2023-10-19T10:37:00.099416+0000","last_undegraded":"2023-10-19T10:37:00.099416+0000","last_fullsized":"2023-10-19T10:37:00.099416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:28:29.050287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096752+0000","last_change":"2023-10-19T10:36:59.358801+0000","last_active":"2023-10-19T10:37:00.096752+0000","last_peered":"2023-10-19T10:37:00.096752+0000","last_clean":"2023-10-19T10:37:00.096752+0000","last_became_active":"2023-10-19T10:36:59.356906+0000","last_became_peered":"2023-10-19T10:36:59.356906+0000","last_unstale":"2023-10-19T10:37:00.096752+0000","last_undegraded":"2023-10-19T10:37:00.096752+0000","last_fullsized":"2023-10-19T10:37:00.096752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:56.099256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097964+0000","last_change":"2023-10-19T10:36:55.105397+0000","last_active":"2023-10-19T10:37:00.097964+0000","last_peered":"2023-10-19T10:37:00.097964+0000","last_clean":"2023-10-19T10:37:00.097964+0000","last_became_active":"2023-10-19T10:36:55.105016+0000","last_became_peered":"2023-10-19T10:36:55.105016+0000","last_unstale":"2023-10-19T10:37:00.097964+0000","last_undegraded":"2023-10-19T10:37:00.097964+0000","last_fullsized":"2023-10-19T10:37:00.097964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:13.604819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323795+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:37:00.323795+0000","last_peered":"2023-10-19T10:37:00.323795+0000","last_clean":"2023-10-19T10:37:00.323795+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:37:00.323795+0000","last_undegraded":"2023-10-19T10:37:00.323795+0000","last_fullsized":"2023-10-19T10:37:00.323795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:47:38.405947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278495+0000","last_change":"2023-10-19T10:36:54.301501+0000","last_active":"2023-10-19T10:37:00.278495+0000","last_peered":"2023-10-19T10:37:00.278495+0000","last_clean":"2023-10-19T10:37:00.278495+0000","last_became_active":"2023-10-19T10:36:54.301193+0000","last_became_peered":"2023-10-19T10:36:54.301193+0000","last_unstale":"2023-10-19T10:37:00.278495+0000","last_undegraded":"2023-10-19T10:37:00.278495+0000","last_fullsized":"2023-10-19T10:37:00.278495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323709+0000","last_change":"2023-10-19T10:36:59.351528+0000","last_active":"2023-10-19T10:37:00.323709+0000","last_peered":"2023-10-19T10:37:00.323709+0000","last_clean":"2023-10-19T10:37:00.323709+0000","last_became_active":"2023-10-19T10:36:59.351102+0000","last_became_peered":"2023-10-19T10:36:59.351102+0000","last_unstale":"2023-10-19T10:37:00.323709+0000","last_undegraded":"2023-10-19T10:37:00.323709+0000","last_fullsized":"2023-10-19T10:37:00.323709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323080+0000","last_change":"2023-10-19T10:36:55.108015+0000","last_active":"2023-10-19T10:37:00.323080+0000","last_peered":"2023-10-19T10:37:00.323080+0000","last_clean":"2023-10-19T10:37:00.323080+0000","last_became_active":"2023-10-19T10:36:55.107693+0000","last_became_peered":"2023-10-19T10:36:55.107693+0000","last_unstale":"2023-10-19T10:37:00.323080+0000","last_undegraded":"2023-10-19T10:37:00.323080+0000","last_fullsized":"2023-10-19T10:37:00.323080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097888+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:37:00.097888+0000","last_peered":"2023-10-19T10:37:00.097888+0000","last_clean":"2023-10-19T10:37:00.097888+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:37:00.097888+0000","last_undegraded":"2023-10-19T10:37:00.097888+0000","last_fullsized":"2023-10-19T10:37:00.097888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:31:56.734648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324508+0000","last_change":"2023-10-19T10:36:54.310814+0000","last_active":"2023-10-19T10:37:00.324508+0000","last_peered":"2023-10-19T10:37:00.324508+0000","last_clean":"2023-10-19T10:37:00.324508+0000","last_became_active":"2023-10-19T10:36:54.310517+0000","last_became_peered":"2023-10-19T10:36:54.310517+0000","last_unstale":"2023-10-19T10:37:00.324508+0000","last_undegraded":"2023-10-19T10:37:00.324508+0000","last_fullsized":"2023-10-19T10:37:00.324508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:37.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097777+0000","last_change":"2023-10-19T10:36:59.353687+0000","last_active":"2023-10-19T10:37:00.097777+0000","last_peered":"2023-10-19T10:37:00.097777+0000","last_clean":"2023-10-19T10:37:00.097777+0000","last_became_active":"2023-10-19T10:36:59.353297+0000","last_became_peered":"2023-10-19T10:36:59.353297+0000","last_unstale":"2023-10-19T10:37:00.097777+0000","last_undegraded":"2023-10-19T10:37:00.097777+0000","last_fullsized":"2023-10-19T10:37:00.097777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:05:36.096009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278152+0000","last_change":"2023-10-19T10:36:55.118741+0000","last_active":"2023-10-19T10:37:00.278152+0000","last_peered":"2023-10-19T10:37:00.278152+0000","last_clean":"2023-10-19T10:37:00.278152+0000","last_became_active":"2023-10-19T10:36:55.116465+0000","last_became_peered":"2023-10-19T10:36:55.116465+0000","last_unstale":"2023-10-19T10:37:00.278152+0000","last_undegraded":"2023-10-19T10:37:00.278152+0000","last_fullsized":"2023-10-19T10:37:00.278152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099045+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:37:00.099045+0000","last_peered":"2023-10-19T10:37:00.099045+0000","last_clean":"2023-10-19T10:37:00.099045+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:37:00.099045+0000","last_undegraded":"2023-10-19T10:37:00.099045+0000","last_fullsized":"2023-10-19T10:37:00.099045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:12.307342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097428+0000","last_change":"2023-10-19T10:36:54.306800+0000","last_active":"2023-10-19T10:37:00.097428+0000","last_peered":"2023-10-19T10:37:00.097428+0000","last_clean":"2023-10-19T10:37:00.097428+0000","last_became_active":"2023-10-19T10:36:54.306507+0000","last_became_peered":"2023-10-19T10:36:54.306507+0000","last_unstale":"2023-10-19T10:37:00.097428+0000","last_undegraded":"2023-10-19T10:37:00.097428+0000","last_fullsized":"2023-10-19T10:37:00.097428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:41.200123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323296+0000","last_change":"2023-10-19T10:36:59.352359+0000","last_active":"2023-10-19T10:37:00.323296+0000","last_peered":"2023-10-19T10:37:00.323296+0000","last_clean":"2023-10-19T10:37:00.323296+0000","last_became_active":"2023-10-19T10:36:59.351975+0000","last_became_peered":"2023-10-19T10:36:59.351975+0000","last_unstale":"2023-10-19T10:37:00.323296+0000","last_undegraded":"2023-10-19T10:37:00.323296+0000","last_fullsized":"2023-10-19T10:37:00.323296+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:39:36.789471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097181+0000","last_change":"2023-10-19T10:36:55.109753+0000","last_active":"2023-10-19T10:37:00.097181+0000","last_peered":"2023-10-19T10:37:00.097181+0000","last_clean":"2023-10-19T10:37:00.097181+0000","last_became_active":"2023-10-19T10:36:55.109424+0000","last_became_peered":"2023-10-19T10:36:55.109424+0000","last_unstale":"2023-10-19T10:37:00.097181+0000","last_undegraded":"2023-10-19T10:37:00.097181+0000","last_fullsized":"2023-10-19T10:37:00.097181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097504+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:37:00.097504+0000","last_peered":"2023-10-19T10:37:00.097504+0000","last_clean":"2023-10-19T10:37:00.097504+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:37:00.097504+0000","last_undegraded":"2023-10-19T10:37:00.097504+0000","last_fullsized":"2023-10-19T10:37:00.097504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:43.871421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324473+0000","last_change":"2023-10-19T10:36:54.305974+0000","last_active":"2023-10-19T10:37:00.324473+0000","last_peered":"2023-10-19T10:37:00.324473+0000","last_clean":"2023-10-19T10:37:00.324473+0000","last_became_active":"2023-10-19T10:36:54.305710+0000","last_became_peered":"2023-10-19T10:36:54.305710+0000","last_unstale":"2023-10-19T10:37:00.324473+0000","last_undegraded":"2023-10-19T10:37:00.324473+0000","last_fullsized":"2023-10-19T10:37:00.324473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277916+0000","last_change":"2023-10-19T10:36:59.353503+0000","last_active":"2023-10-19T10:37:00.277916+0000","last_peered":"2023-10-19T10:37:00.277916+0000","last_clean":"2023-10-19T10:37:00.277916+0000","last_became_active":"2023-10-19T10:36:59.352985+0000","last_became_peered":"2023-10-19T10:36:59.352985+0000","last_unstale":"2023-10-19T10:37:00.277916+0000","last_undegraded":"2023-10-19T10:37:00.277916+0000","last_fullsized":"2023-10-19T10:37:00.277916+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097415+0000","last_change":"2023-10-19T10:36:55.112577+0000","last_active":"2023-10-19T10:37:00.097415+0000","last_peered":"2023-10-19T10:37:00.097415+0000","last_clean":"2023-10-19T10:37:00.097415+0000","last_became_active":"2023-10-19T10:36:55.112302+0000","last_became_peered":"2023-10-19T10:36:55.112302+0000","last_unstale":"2023-10-19T10:37:00.097415+0000","last_undegraded":"2023-10-19T10:37:00.097415+0000","last_fullsized":"2023-10-19T10:37:00.097415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099330+0000","last_change":"2023-10-19T10:36:54.302990+0000","last_active":"2023-10-19T10:37:00.099330+0000","last_peered":"2023-10-19T10:37:00.099330+0000","last_clean":"2023-10-19T10:37:00.099330+0000","last_became_active":"2023-10-19T10:36:54.302553+0000","last_became_peered":"2023-10-19T10:36:54.302553+0000","last_unstale":"2023-10-19T10:37:00.099330+0000","last_undegraded":"2023-10-19T10:37:00.099330+0000","last_fullsized":"2023-10-19T10:37:00.099330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:07:06.761773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323861+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:37:00.323861+0000","last_peered":"2023-10-19T10:37:00.323861+0000","last_clean":"2023-10-19T10:37:00.323861+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:37:00.323861+0000","last_undegraded":"2023-10-19T10:37:00.323861+0000","last_fullsized":"2023-10-19T10:37:00.323861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:37.072096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277993+0000","last_change":"2023-10-19T10:36:59.351199+0000","last_active":"2023-10-19T10:37:00.277993+0000","last_peered":"2023-10-19T10:37:00.277993+0000","last_clean":"2023-10-19T10:37:00.277993+0000","last_became_active":"2023-10-19T10:36:59.350722+0000","last_became_peered":"2023-10-19T10:36:59.350722+0000","last_unstale":"2023-10-19T10:37:00.277993+0000","last_undegraded":"2023-10-19T10:37:00.277993+0000","last_fullsized":"2023-10-19T10:37:00.277993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097884+0000","last_change":"2023-10-19T10:36:55.108041+0000","last_active":"2023-10-19T10:37:00.097884+0000","last_peered":"2023-10-19T10:37:00.097884+0000","last_clean":"2023-10-19T10:37:00.097884+0000","last_became_active":"2023-10-19T10:36:55.107747+0000","last_became_peered":"2023-10-19T10:36:55.107747+0000","last_unstale":"2023-10-19T10:37:00.097884+0000","last_undegraded":"2023-10-19T10:37:00.097884+0000","last_fullsized":"2023-10-19T10:37:00.097884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096738+0000","last_change":"2023-10-19T10:36:54.307533+0000","last_active":"2023-10-19T10:37:00.096738+0000","last_peered":"2023-10-19T10:37:00.096738+0000","last_clean":"2023-10-19T10:37:00.096738+0000","last_became_active":"2023-10-19T10:36:54.307237+0000","last_became_peered":"2023-10-19T10:36:54.307237+0000","last_unstale":"2023-10-19T10:37:00.096738+0000","last_undegraded":"2023-10-19T10:37:00.096738+0000","last_fullsized":"2023-10-19T10:37:00.096738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:57.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322887+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:37:00.322887+0000","last_peered":"2023-10-19T10:37:00.322887+0000","last_clean":"2023-10-19T10:37:00.322887+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:37:00.322887+0000","last_undegraded":"2023-10-19T10:37:00.322887+0000","last_fullsized":"2023-10-19T10:37:00.322887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:35:30.635889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324367+0000","last_change":"2023-10-19T10:36:59.349089+0000","last_active":"2023-10-19T10:37:00.324367+0000","last_peered":"2023-10-19T10:37:00.324367+0000","last_clean":"2023-10-19T10:37:00.324367+0000","last_became_active":"2023-10-19T10:36:59.348703+0000","last_became_peered":"2023-10-19T10:36:59.348703+0000","last_unstale":"2023-10-19T10:37:00.324367+0000","last_undegraded":"2023-10-19T10:37:00.324367+0000","last_fullsized":"2023-10-19T10:37:00.324367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278271+0000","last_change":"2023-10-19T10:36:55.112250+0000","last_active":"2023-10-19T10:37:00.278271+0000","last_peered":"2023-10-19T10:37:00.278271+0000","last_clean":"2023-10-19T10:37:00.278271+0000","last_became_active":"2023-10-19T10:36:55.111996+0000","last_became_peered":"2023-10-19T10:36:55.111996+0000","last_unstale":"2023-10-19T10:37:00.278271+0000","last_undegraded":"2023-10-19T10:37:00.278271+0000","last_fullsized":"2023-10-19T10:37:00.278271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277550+0000","last_change":"2023-10-19T10:36:54.307098+0000","last_active":"2023-10-19T10:37:00.277550+0000","last_peered":"2023-10-19T10:37:00.277550+0000","last_clean":"2023-10-19T10:37:00.277550+0000","last_became_active":"2023-10-19T10:36:54.306769+0000","last_became_peered":"2023-10-19T10:36:54.306769+0000","last_unstale":"2023-10-19T10:37:00.277550+0000","last_undegraded":"2023-10-19T10:37:00.277550+0000","last_fullsized":"2023-10-19T10:37:00.277550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:13.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098430+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:37:00.098430+0000","last_peered":"2023-10-19T10:37:00.098430+0000","last_clean":"2023-10-19T10:37:00.098430+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:37:00.098430+0000","last_undegraded":"2023-10-19T10:37:00.098430+0000","last_fullsized":"2023-10-19T10:37:00.098430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324610+0000","last_change":"2023-10-19T10:36:59.357456+0000","last_active":"2023-10-19T10:37:00.324610+0000","last_peered":"2023-10-19T10:37:00.324610+0000","last_clean":"2023-10-19T10:37:00.324610+0000","last_became_active":"2023-10-19T10:36:59.357079+0000","last_became_peered":"2023-10-19T10:36:59.357079+0000","last_unstale":"2023-10-19T10:37:00.324610+0000","last_undegraded":"2023-10-19T10:37:00.324610+0000","last_fullsized":"2023-10-19T10:37:00.324610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:52.409495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323740+0000","last_change":"2023-10-19T10:36:55.113749+0000","last_active":"2023-10-19T10:37:00.323740+0000","last_peered":"2023-10-19T10:37:00.323740+0000","last_clean":"2023-10-19T10:37:00.323740+0000","last_became_active":"2023-10-19T10:36:55.113459+0000","last_became_peered":"2023-10-19T10:36:55.113459+0000","last_unstale":"2023-10-19T10:37:00.323740+0000","last_undegraded":"2023-10-19T10:37:00.323740+0000","last_fullsized":"2023-10-19T10:37:00.323740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:05:40.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323207+0000","last_change":"2023-10-19T10:36:54.309423+0000","last_active":"2023-10-19T10:37:00.323207+0000","last_peered":"2023-10-19T10:37:00.323207+0000","last_clean":"2023-10-19T10:37:00.323207+0000","last_became_active":"2023-10-19T10:36:54.309122+0000","last_became_peered":"2023-10-19T10:36:54.309122+0000","last_unstale":"2023-10-19T10:37:00.323207+0000","last_undegraded":"2023-10-19T10:37:00.323207+0000","last_fullsized":"2023-10-19T10:37:00.323207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:47.999447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324166+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:37:00.324166+0000","last_peered":"2023-10-19T10:37:00.324166+0000","last_clean":"2023-10-19T10:37:00.324166+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:37:00.324166+0000","last_undegraded":"2023-10-19T10:37:00.324166+0000","last_fullsized":"2023-10-19T10:37:00.324166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098587+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:37:00.098587+0000","last_peered":"2023-10-19T10:37:00.098587+0000","last_clean":"2023-10-19T10:37:00.098587+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:37:00.098587+0000","last_undegraded":"2023-10-19T10:37:00.098587+0000","last_fullsized":"2023-10-19T10:37:00.098587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:14.745066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098140+0000","last_change":"2023-10-19T10:36:59.359383+0000","last_active":"2023-10-19T10:37:00.098140+0000","last_peered":"2023-10-19T10:37:00.098140+0000","last_clean":"2023-10-19T10:37:00.098140+0000","last_became_active":"2023-10-19T10:36:59.358978+0000","last_became_peered":"2023-10-19T10:36:59.358978+0000","last_unstale":"2023-10-19T10:37:00.098140+0000","last_undegraded":"2023-10-19T10:37:00.098140+0000","last_fullsized":"2023-10-19T10:37:00.098140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097806+0000","last_change":"2023-10-19T10:36:55.111833+0000","last_active":"2023-10-19T10:37:00.097806+0000","last_peered":"2023-10-19T10:37:00.097806+0000","last_clean":"2023-10-19T10:37:00.097806+0000","last_became_active":"2023-10-19T10:36:55.111558+0000","last_became_peered":"2023-10-19T10:36:55.111558+0000","last_unstale":"2023-10-19T10:37:00.097806+0000","last_undegraded":"2023-10-19T10:37:00.097806+0000","last_fullsized":"2023-10-19T10:37:00.097806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323791+0000","last_change":"2023-10-19T10:36:54.304575+0000","last_active":"2023-10-19T10:37:00.323791+0000","last_peered":"2023-10-19T10:37:00.323791+0000","last_clean":"2023-10-19T10:37:00.323791+0000","last_became_active":"2023-10-19T10:36:54.304001+0000","last_became_peered":"2023-10-19T10:36:54.304001+0000","last_unstale":"2023-10-19T10:37:00.323791+0000","last_undegraded":"2023-10-19T10:37:00.323791+0000","last_fullsized":"2023-10-19T10:37:00.323791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099327+0000","last_change":"2023-10-19T10:36:59.347266+0000","last_active":"2023-10-19T10:37:00.099327+0000","last_peered":"2023-10-19T10:37:00.099327+0000","last_clean":"2023-10-19T10:37:00.099327+0000","last_became_active":"2023-10-19T10:36:59.346858+0000","last_became_peered":"2023-10-19T10:36:59.346858+0000","last_unstale":"2023-10-19T10:37:00.099327+0000","last_undegraded":"2023-10-19T10:37:00.099327+0000","last_fullsized":"2023-10-19T10:37:00.099327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:30.546568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099297+0000","last_change":"2023-10-19T10:36:55.107822+0000","last_active":"2023-10-19T10:37:00.099297+0000","last_peered":"2023-10-19T10:37:00.099297+0000","last_clean":"2023-10-19T10:37:00.099297+0000","last_became_active":"2023-10-19T10:36:55.107534+0000","last_became_peered":"2023-10-19T10:36:55.107534+0000","last_unstale":"2023-10-19T10:37:00.099297+0000","last_undegraded":"2023-10-19T10:37:00.099297+0000","last_fullsized":"2023-10-19T10:37:00.099297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:51:56.439816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098149+0000","last_change":"2023-10-19T10:36:54.306612+0000","last_active":"2023-10-19T10:37:00.098149+0000","last_peered":"2023-10-19T10:37:00.098149+0000","last_clean":"2023-10-19T10:37:00.098149+0000","last_became_active":"2023-10-19T10:36:54.306357+0000","last_became_peered":"2023-10-19T10:36:54.306357+0000","last_unstale":"2023-10-19T10:37:00.098149+0000","last_undegraded":"2023-10-19T10:37:00.098149+0000","last_fullsized":"2023-10-19T10:37:00.098149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:37:13.605874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324569+0000","last_change":"2023-10-19T10:36:59.352368+0000","last_active":"2023-10-19T10:37:00.324569+0000","last_peered":"2023-10-19T10:37:00.324569+0000","last_clean":"2023-10-19T10:37:00.324569+0000","last_became_active":"2023-10-19T10:36:59.352002+0000","last_became_peered":"2023-10-19T10:36:59.352002+0000","last_unstale":"2023-10-19T10:37:00.324569+0000","last_undegraded":"2023-10-19T10:37:00.324569+0000","last_fullsized":"2023-10-19T10:37:00.324569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:05.263646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099519+0000","last_change":"2023-10-19T10:36:55.110179+0000","last_active":"2023-10-19T10:37:00.099519+0000","last_peered":"2023-10-19T10:37:00.099519+0000","last_clean":"2023-10-19T10:37:00.099519+0000","last_became_active":"2023-10-19T10:36:55.109756+0000","last_became_peered":"2023-10-19T10:36:55.109756+0000","last_unstale":"2023-10-19T10:37:00.099519+0000","last_undegraded":"2023-10-19T10:37:00.099519+0000","last_fullsized":"2023-10-19T10:37:00.099519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:55:21.657235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323576+0000","last_change":"2023-10-19T10:36:54.306427+0000","last_active":"2023-10-19T10:37:00.323576+0000","last_peered":"2023-10-19T10:37:00.323576+0000","last_clean":"2023-10-19T10:37:00.323576+0000","last_became_active":"2023-10-19T10:36:54.306033+0000","last_became_peered":"2023-10-19T10:36:54.306033+0000","last_unstale":"2023-10-19T10:37:00.323576+0000","last_undegraded":"2023-10-19T10:37:00.323576+0000","last_fullsized":"2023-10-19T10:37:00.323576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:08:11.353560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097356+0000","last_change":"2023-10-19T10:36:59.353562+0000","last_active":"2023-10-19T10:37:00.097356+0000","last_peered":"2023-10-19T10:37:00.097356+0000","last_clean":"2023-10-19T10:37:00.097356+0000","last_became_active":"2023-10-19T10:36:59.353204+0000","last_became_peered":"2023-10-19T10:36:59.353204+0000","last_unstale":"2023-10-19T10:37:00.097356+0000","last_undegraded":"2023-10-19T10:37:00.097356+0000","last_fullsized":"2023-10-19T10:37:00.097356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099281+0000","last_change":"2023-10-19T10:36:55.115534+0000","last_active":"2023-10-19T10:37:00.099281+0000","last_peered":"2023-10-19T10:37:00.099281+0000","last_clean":"2023-10-19T10:37:00.099281+0000","last_became_active":"2023-10-19T10:36:55.114049+0000","last_became_peered":"2023-10-19T10:36:55.114049+0000","last_unstale":"2023-10-19T10:37:00.099281+0000","last_undegraded":"2023-10-19T10:37:00.099281+0000","last_fullsized":"2023-10-19T10:37:00.099281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:22:31.265043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323955+0000","last_change":"2023-10-19T10:36:54.310069+0000","last_active":"2023-10-19T10:37:00.323955+0000","last_peered":"2023-10-19T10:37:00.323955+0000","last_clean":"2023-10-19T10:37:00.323955+0000","last_became_active":"2023-10-19T10:36:54.309784+0000","last_became_peered":"2023-10-19T10:36:54.309784+0000","last_unstale":"2023-10-19T10:37:00.323955+0000","last_undegraded":"2023-10-19T10:37:00.323955+0000","last_fullsized":"2023-10-19T10:37:00.323955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323321+0000","last_change":"2023-10-19T10:36:59.351003+0000","last_active":"2023-10-19T10:37:00.323321+0000","last_peered":"2023-10-19T10:37:00.323321+0000","last_clean":"2023-10-19T10:37:00.323321+0000","last_became_active":"2023-10-19T10:36:59.349597+0000","last_became_peered":"2023-10-19T10:36:59.349597+0000","last_unstale":"2023-10-19T10:37:00.323321+0000","last_undegraded":"2023-10-19T10:37:00.323321+0000","last_fullsized":"2023-10-19T10:37:00.323321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:53:25.552100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277872+0000","last_change":"2023-10-19T10:36:55.118959+0000","last_active":"2023-10-19T10:37:00.277872+0000","last_peered":"2023-10-19T10:37:00.277872+0000","last_clean":"2023-10-19T10:37:00.277872+0000","last_became_active":"2023-10-19T10:36:55.116867+0000","last_became_peered":"2023-10-19T10:36:55.116867+0000","last_unstale":"2023-10-19T10:37:00.277872+0000","last_undegraded":"2023-10-19T10:37:00.277872+0000","last_fullsized":"2023-10-19T10:37:00.277872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324124+0000","last_change":"2023-10-19T10:36:54.305145+0000","last_active":"2023-10-19T10:37:00.324124+0000","last_peered":"2023-10-19T10:37:00.324124+0000","last_clean":"2023-10-19T10:37:00.324124+0000","last_became_active":"2023-10-19T10:36:54.304599+0000","last_became_peered":"2023-10-19T10:36:54.304599+0000","last_unstale":"2023-10-19T10:37:00.324124+0000","last_undegraded":"2023-10-19T10:37:00.324124+0000","last_fullsized":"2023-10-19T10:37:00.324124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323492+0000","last_change":"2023-10-19T10:36:59.348091+0000","last_active":"2023-10-19T10:37:00.323492+0000","last_peered":"2023-10-19T10:37:00.323492+0000","last_clean":"2023-10-19T10:37:00.323492+0000","last_became_active":"2023-10-19T10:36:59.346180+0000","last_became_peered":"2023-10-19T10:36:59.346180+0000","last_unstale":"2023-10-19T10:37:00.323492+0000","last_undegraded":"2023-10-19T10:37:00.323492+0000","last_fullsized":"2023-10-19T10:37:00.323492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:42:00.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323299+0000","last_change":"2023-10-19T10:36:55.114648+0000","last_active":"2023-10-19T10:37:00.323299+0000","last_peered":"2023-10-19T10:37:00.323299+0000","last_clean":"2023-10-19T10:37:00.323299+0000","last_became_active":"2023-10-19T10:36:55.114149+0000","last_became_peered":"2023-10-19T10:36:55.114149+0000","last_unstale":"2023-10-19T10:37:00.323299+0000","last_undegraded":"2023-10-19T10:37:00.323299+0000","last_fullsized":"2023-10-19T10:37:00.323299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:25:33.396338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098964+0000","last_change":"2023-10-19T10:36:54.309069+0000","last_active":"2023-10-19T10:37:00.098964+0000","last_peered":"2023-10-19T10:37:00.098964+0000","last_clean":"2023-10-19T10:37:00.098964+0000","last_became_active":"2023-10-19T10:36:54.308767+0000","last_became_peered":"2023-10-19T10:36:54.308767+0000","last_unstale":"2023-10-19T10:37:00.098964+0000","last_undegraded":"2023-10-19T10:37:00.098964+0000","last_fullsized":"2023-10-19T10:37:00.098964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323054+0000","last_change":"2023-10-19T10:36:59.349735+0000","last_active":"2023-10-19T10:37:00.323054+0000","last_peered":"2023-10-19T10:37:00.323054+0000","last_clean":"2023-10-19T10:37:00.323054+0000","last_became_active":"2023-10-19T10:36:59.349327+0000","last_became_peered":"2023-10-19T10:36:59.349327+0000","last_unstale":"2023-10-19T10:37:00.323054+0000","last_undegraded":"2023-10-19T10:37:00.323054+0000","last_fullsized":"2023-10-19T10:37:00.323054+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:51:57.082676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097759+0000","last_change":"2023-10-19T10:36:55.111295+0000","last_active":"2023-10-19T10:37:00.097759+0000","last_peered":"2023-10-19T10:37:00.097759+0000","last_clean":"2023-10-19T10:37:00.097759+0000","last_became_active":"2023-10-19T10:36:55.111016+0000","last_became_peered":"2023-10-19T10:36:55.111016+0000","last_unstale":"2023-10-19T10:37:00.097759+0000","last_undegraded":"2023-10-19T10:37:00.097759+0000","last_fullsized":"2023-10-19T10:37:00.097759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:01:05.121484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098592+0000","last_change":"2023-10-19T10:36:54.308800+0000","last_active":"2023-10-19T10:37:00.098592+0000","last_peered":"2023-10-19T10:37:00.098592+0000","last_clean":"2023-10-19T10:37:00.098592+0000","last_became_active":"2023-10-19T10:36:54.307919+0000","last_became_peered":"2023-10-19T10:36:54.307919+0000","last_unstale":"2023-10-19T10:37:00.098592+0000","last_undegraded":"2023-10-19T10:37:00.098592+0000","last_fullsized":"2023-10-19T10:37:00.098592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:00.862675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322763+0000","last_change":"2023-10-19T10:36:59.350698+0000","last_active":"2023-10-19T10:37:00.322763+0000","last_peered":"2023-10-19T10:37:00.322763+0000","last_clean":"2023-10-19T10:37:00.322763+0000","last_became_active":"2023-10-19T10:36:59.350331+0000","last_became_peered":"2023-10-19T10:36:59.350331+0000","last_unstale":"2023-10-19T10:37:00.322763+0000","last_undegraded":"2023-10-19T10:37:00.322763+0000","last_fullsized":"2023-10-19T10:37:00.322763+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:44.911501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323455+0000","last_change":"2023-10-19T10:36:55.110357+0000","last_active":"2023-10-19T10:37:00.323455+0000","last_peered":"2023-10-19T10:37:00.323455+0000","last_clean":"2023-10-19T10:37:00.323455+0000","last_became_active":"2023-10-19T10:36:55.110058+0000","last_became_peered":"2023-10-19T10:36:55.110058+0000","last_unstale":"2023-10-19T10:37:00.323455+0000","last_undegraded":"2023-10-19T10:37:00.323455+0000","last_fullsized":"2023-10-19T10:37:00.323455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:39.760467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098977+0000","last_change":"2023-10-19T10:36:54.305854+0000","last_active":"2023-10-19T10:37:00.098977+0000","last_peered":"2023-10-19T10:37:00.098977+0000","last_clean":"2023-10-19T10:37:00.098977+0000","last_became_active":"2023-10-19T10:36:54.305559+0000","last_became_peered":"2023-10-19T10:36:54.305559+0000","last_unstale":"2023-10-19T10:37:00.098977+0000","last_undegraded":"2023-10-19T10:37:00.098977+0000","last_fullsized":"2023-10-19T10:37:00.098977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097838+0000","last_change":"2023-10-19T10:36:59.359644+0000","last_active":"2023-10-19T10:37:00.097838+0000","last_peered":"2023-10-19T10:37:00.097838+0000","last_clean":"2023-10-19T10:37:00.097838+0000","last_became_active":"2023-10-19T10:36:59.359083+0000","last_became_peered":"2023-10-19T10:36:59.359083+0000","last_unstale":"2023-10-19T10:37:00.097838+0000","last_undegraded":"2023-10-19T10:37:00.097838+0000","last_fullsized":"2023-10-19T10:37:00.097838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324477+0000","last_change":"2023-10-19T10:36:59.349191+0000","last_active":"2023-10-19T10:37:00.324477+0000","last_peered":"2023-10-19T10:37:00.324477+0000","last_clean":"2023-10-19T10:37:00.324477+0000","last_became_active":"2023-10-19T10:36:59.348388+0000","last_became_peered":"2023-10-19T10:36:59.348388+0000","last_unstale":"2023-10-19T10:37:00.324477+0000","last_undegraded":"2023-10-19T10:37:00.324477+0000","last_fullsized":"2023-10-19T10:37:00.324477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323413+0000","last_change":"2023-10-19T10:36:54.311108+0000","last_active":"2023-10-19T10:37:00.323413+0000","last_peered":"2023-10-19T10:37:00.323413+0000","last_clean":"2023-10-19T10:37:00.323413+0000","last_became_active":"2023-10-19T10:36:54.310804+0000","last_became_peered":"2023-10-19T10:36:54.310804+0000","last_unstale":"2023-10-19T10:37:00.323413+0000","last_undegraded":"2023-10-19T10:37:00.323413+0000","last_fullsized":"2023-10-19T10:37:00.323413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:42:24.153777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324424+0000","last_change":"2023-10-19T10:36:55.108985+0000","last_active":"2023-10-19T10:37:00.324424+0000","last_peered":"2023-10-19T10:37:00.324424+0000","last_clean":"2023-10-19T10:37:00.324424+0000","last_became_active":"2023-10-19T10:36:55.108726+0000","last_became_peered":"2023-10-19T10:36:55.108726+0000","last_unstale":"2023-10-19T10:37:00.324424+0000","last_undegraded":"2023-10-19T10:37:00.324424+0000","last_fullsized":"2023-10-19T10:37:00.324424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:29.622437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324136+0000","last_change":"2023-10-19T10:36:59.352844+0000","last_active":"2023-10-19T10:37:00.324136+0000","last_peered":"2023-10-19T10:37:00.324136+0000","last_clean":"2023-10-19T10:37:00.324136+0000","last_became_active":"2023-10-19T10:36:59.352451+0000","last_became_peered":"2023-10-19T10:36:59.352451+0000","last_unstale":"2023-10-19T10:37:00.324136+0000","last_undegraded":"2023-10-19T10:37:00.324136+0000","last_fullsized":"2023-10-19T10:37:00.324136+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:21.477523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098487+0000","last_change":"2023-10-19T10:36:54.310021+0000","last_active":"2023-10-19T10:37:00.098487+0000","last_peered":"2023-10-19T10:37:00.098487+0000","last_clean":"2023-10-19T10:37:00.098487+0000","last_became_active":"2023-10-19T10:36:54.309686+0000","last_became_peered":"2023-10-19T10:36:54.309686+0000","last_unstale":"2023-10-19T10:37:00.098487+0000","last_undegraded":"2023-10-19T10:37:00.098487+0000","last_fullsized":"2023-10-19T10:37:00.098487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:25.891199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278441+0000","last_change":"2023-10-19T10:36:55.118465+0000","last_active":"2023-10-19T10:37:00.278441+0000","last_peered":"2023-10-19T10:37:00.278441+0000","last_clean":"2023-10-19T10:37:00.278441+0000","last_became_active":"2023-10-19T10:36:55.116202+0000","last_became_peered":"2023-10-19T10:36:55.116202+0000","last_unstale":"2023-10-19T10:37:00.278441+0000","last_undegraded":"2023-10-19T10:37:00.278441+0000","last_fullsized":"2023-10-19T10:37:00.278441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277600+0000","last_change":"2023-10-19T10:36:54.306522+0000","last_active":"2023-10-19T10:37:00.277600+0000","last_peered":"2023-10-19T10:37:00.277600+0000","last_clean":"2023-10-19T10:37:00.277600+0000","last_became_active":"2023-10-19T10:36:54.306147+0000","last_became_peered":"2023-10-19T10:36:54.306147+0000","last_unstale":"2023-10-19T10:37:00.277600+0000","last_undegraded":"2023-10-19T10:37:00.277600+0000","last_fullsized":"2023-10-19T10:37:00.277600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:38.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324371+0000","last_change":"2023-10-19T10:36:55.110979+0000","last_active":"2023-10-19T10:37:00.324371+0000","last_peered":"2023-10-19T10:37:00.324371+0000","last_clean":"2023-10-19T10:37:00.324371+0000","last_became_active":"2023-10-19T10:36:55.109937+0000","last_became_peered":"2023-10-19T10:36:55.109937+0000","last_unstale":"2023-10-19T10:37:00.324371+0000","last_undegraded":"2023-10-19T10:37:00.324371+0000","last_fullsized":"2023-10-19T10:37:00.324371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323485+0000","last_change":"2023-10-19T10:36:59.355501+0000","last_active":"2023-10-19T10:37:00.323485+0000","last_peered":"2023-10-19T10:37:00.323485+0000","last_clean":"2023-10-19T10:37:00.323485+0000","last_became_active":"2023-10-19T10:36:59.353627+0000","last_became_peered":"2023-10-19T10:36:59.353627+0000","last_unstale":"2023-10-19T10:37:00.323485+0000","last_undegraded":"2023-10-19T10:37:00.323485+0000","last_fullsized":"2023-10-19T10:37:00.323485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:30.933353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098816+0000","last_change":"2023-10-19T10:36:54.308041+0000","last_active":"2023-10-19T10:37:00.098816+0000","last_peered":"2023-10-19T10:37:00.098816+0000","last_clean":"2023-10-19T10:37:00.098816+0000","last_became_active":"2023-10-19T10:36:54.307752+0000","last_became_peered":"2023-10-19T10:36:54.307752+0000","last_unstale":"2023-10-19T10:37:00.098816+0000","last_undegraded":"2023-10-19T10:37:00.098816+0000","last_fullsized":"2023-10-19T10:37:00.098816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323551+0000","last_change":"2023-10-19T10:36:55.115058+0000","last_active":"2023-10-19T10:37:00.323551+0000","last_peered":"2023-10-19T10:37:00.323551+0000","last_clean":"2023-10-19T10:37:00.323551+0000","last_became_active":"2023-10-19T10:36:55.114729+0000","last_became_peered":"2023-10-19T10:36:55.114729+0000","last_unstale":"2023-10-19T10:37:00.323551+0000","last_undegraded":"2023-10-19T10:37:00.323551+0000","last_fullsized":"2023-10-19T10:37:00.323551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324110+0000","last_change":"2023-10-19T10:36:59.356762+0000","last_active":"2023-10-19T10:37:00.324110+0000","last_peered":"2023-10-19T10:37:00.324110+0000","last_clean":"2023-10-19T10:37:00.324110+0000","last_became_active":"2023-10-19T10:36:59.356422+0000","last_became_peered":"2023-10-19T10:36:59.356422+0000","last_unstale":"2023-10-19T10:37:00.324110+0000","last_undegraded":"2023-10-19T10:37:00.324110+0000","last_fullsized":"2023-10-19T10:37:00.324110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323028+0000","last_change":"2023-10-19T10:36:55.108618+0000","last_active":"2023-10-19T10:37:00.323028+0000","last_peered":"2023-10-19T10:37:00.323028+0000","last_clean":"2023-10-19T10:37:00.323028+0000","last_became_active":"2023-10-19T10:36:55.108364+0000","last_became_peered":"2023-10-19T10:36:55.108364+0000","last_unstale":"2023-10-19T10:37:00.323028+0000","last_undegraded":"2023-10-19T10:37:00.323028+0000","last_fullsized":"2023-10-19T10:37:00.323028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323784+0000","last_change":"2023-10-19T10:36:55.114936+0000","last_active":"2023-10-19T10:37:00.323784+0000","last_peered":"2023-10-19T10:37:00.323784+0000","last_clean":"2023-10-19T10:37:00.323784+0000","last_became_active":"2023-10-19T10:36:55.114673+0000","last_became_peered":"2023-10-19T10:36:55.114673+0000","last_unstale":"2023-10-19T10:37:00.323784+0000","last_undegraded":"2023-10-19T10:37:00.323784+0000","last_fullsized":"2023-10-19T10:37:00.323784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098152+0000","last_change":"2023-10-19T10:36:55.107446+0000","last_active":"2023-10-19T10:37:00.098152+0000","last_peered":"2023-10-19T10:37:00.098152+0000","last_clean":"2023-10-19T10:37:00.098152+0000","last_became_active":"2023-10-19T10:36:55.107185+0000","last_became_peered":"2023-10-19T10:36:55.107185+0000","last_unstale":"2023-10-19T10:37:00.098152+0000","last_undegraded":"2023-10-19T10:37:00.098152+0000","last_fullsized":"2023-10-19T10:37:00.098152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:36:13.224759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323701+0000","last_change":"2023-10-19T10:36:55.108586+0000","last_active":"2023-10-19T10:37:00.323701+0000","last_peered":"2023-10-19T10:37:00.323701+0000","last_clean":"2023-10-19T10:37:00.323701+0000","last_became_active":"2023-10-19T10:36:55.108308+0000","last_became_peered":"2023-10-19T10:36:55.108308+0000","last_unstale":"2023-10-19T10:37:00.323701+0000","last_undegraded":"2023-10-19T10:37:00.323701+0000","last_fullsized":"2023-10-19T10:37:00.323701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:09.378639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097485+0000","last_change":"2023-10-19T10:36:55.112395+0000","last_active":"2023-10-19T10:37:00.097485+0000","last_peered":"2023-10-19T10:37:00.097485+0000","last_clean":"2023-10-19T10:37:00.097485+0000","last_became_active":"2023-10-19T10:36:55.112101+0000","last_became_peered":"2023-10-19T10:36:55.112101+0000","last_unstale":"2023-10-19T10:37:00.097485+0000","last_undegraded":"2023-10-19T10:37:00.097485+0000","last_fullsized":"2023-10-19T10:37:00.097485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:55.114054+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:55.113749+0000","last_became_peered":"2023-10-19T10:36:55.113749+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097817+0000","last_change":"2023-10-19T10:36:55.109364+0000","last_active":"2023-10-19T10:37:00.097817+0000","last_peered":"2023-10-19T10:37:00.097817+0000","last_clean":"2023-10-19T10:37:00.097817+0000","last_became_active":"2023-10-19T10:36:55.109110+0000","last_became_peered":"2023-10-19T10:36:55.109110+0000","last_unstale":"2023-10-19T10:37:00.097817+0000","last_undegraded":"2023-10-19T10:37:00.097817+0000","last_fullsized":"2023-10-19T10:37:00.097817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097790+0000","last_change":"2023-10-19T10:36:55.111296+0000","last_active":"2023-10-19T10:37:00.097790+0000","last_peered":"2023-10-19T10:37:00.097790+0000","last_clean":"2023-10-19T10:37:00.097790+0000","last_became_active":"2023-10-19T10:36:55.111042+0000","last_became_peered":"2023-10-19T10:36:55.111042+0000","last_unstale":"2023-10-19T10:37:00.097790+0000","last_undegraded":"2023-10-19T10:37:00.097790+0000","last_fullsized":"2023-10-19T10:37:00.097790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098307+0000","last_change":"2023-10-19T10:36:55.117568+0000","last_active":"2023-10-19T10:37:00.098307+0000","last_peered":"2023-10-19T10:37:00.098307+0000","last_clean":"2023-10-19T10:37:00.098307+0000","last_became_active":"2023-10-19T10:36:55.117281+0000","last_became_peered":"2023-10-19T10:36:55.117281+0000","last_unstale":"2023-10-19T10:37:00.098307+0000","last_undegraded":"2023-10-19T10:37:00.098307+0000","last_fullsized":"2023-10-19T10:37:00.098307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323081+0000","last_change":"2023-10-19T10:36:55.114123+0000","last_active":"2023-10-19T10:37:00.323081+0000","last_peered":"2023-10-19T10:37:00.323081+0000","last_clean":"2023-10-19T10:37:00.323081+0000","last_became_active":"2023-10-19T10:36:55.113832+0000","last_became_peered":"2023-10-19T10:36:55.113832+0000","last_unstale":"2023-10-19T10:37:00.323081+0000","last_undegraded":"2023-10-19T10:37:00.323081+0000","last_fullsized":"2023-10-19T10:37:00.323081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277576+0000","last_change":"2023-10-19T10:36:55.115474+0000","last_active":"2023-10-19T10:37:00.277576+0000","last_peered":"2023-10-19T10:37:00.277576+0000","last_clean":"2023-10-19T10:37:00.277576+0000","last_became_active":"2023-10-19T10:36:55.115133+0000","last_became_peered":"2023-10-19T10:36:55.115133+0000","last_unstale":"2023-10-19T10:37:00.277576+0000","last_undegraded":"2023-10-19T10:37:00.277576+0000","last_fullsized":"2023-10-19T10:37:00.277576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098770+0000","last_change":"2023-10-19T10:36:55.117512+0000","last_active":"2023-10-19T10:37:00.098770+0000","last_peered":"2023-10-19T10:37:00.098770+0000","last_clean":"2023-10-19T10:37:00.098770+0000","last_became_active":"2023-10-19T10:36:55.117143+0000","last_became_peered":"2023-10-19T10:36:55.117143+0000","last_unstale":"2023-10-19T10:37:00.098770+0000","last_undegraded":"2023-10-19T10:37:00.098770+0000","last_fullsized":"2023-10-19T10:37:00.098770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:46.891573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099100+0000","last_change":"2023-10-19T10:36:55.106989+0000","last_active":"2023-10-19T10:37:00.099100+0000","last_peered":"2023-10-19T10:37:00.099100+0000","last_clean":"2023-10-19T10:37:00.099100+0000","last_became_active":"2023-10-19T10:36:55.106735+0000","last_became_peered":"2023-10-19T10:36:55.106735+0000","last_unstale":"2023-10-19T10:37:00.099100+0000","last_undegraded":"2023-10-19T10:37:00.099100+0000","last_fullsized":"2023-10-19T10:37:00.099100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:49:54.003217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324208+0000","last_change":"2023-10-19T10:36:55.107156+0000","last_active":"2023-10-19T10:37:00.324208+0000","last_peered":"2023-10-19T10:37:00.324208+0000","last_clean":"2023-10-19T10:37:00.324208+0000","last_became_active":"2023-10-19T10:36:55.106636+0000","last_became_peered":"2023-10-19T10:36:55.106636+0000","last_unstale":"2023-10-19T10:37:00.324208+0000","last_undegraded":"2023-10-19T10:37:00.324208+0000","last_fullsized":"2023-10-19T10:37:00.324208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:02:38.159925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098995+0000","last_change":"2023-10-19T10:36:55.102460+0000","last_active":"2023-10-19T10:37:00.098995+0000","last_peered":"2023-10-19T10:37:00.098995+0000","last_clean":"2023-10-19T10:37:00.098995+0000","last_became_active":"2023-10-19T10:36:55.102197+0000","last_became_peered":"2023-10-19T10:36:55.102197+0000","last_unstale":"2023-10-19T10:37:00.098995+0000","last_undegraded":"2023-10-19T10:37:00.098995+0000","last_fullsized":"2023-10-19T10:37:00.098995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:18:06.083802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278315+0000","last_change":"2023-10-19T10:36:55.118142+0000","last_active":"2023-10-19T10:37:00.278315+0000","last_peered":"2023-10-19T10:37:00.278315+0000","last_clean":"2023-10-19T10:37:00.278315+0000","last_became_active":"2023-10-19T10:36:55.116529+0000","last_became_peered":"2023-10-19T10:36:55.116529+0000","last_unstale":"2023-10-19T10:37:00.278315+0000","last_undegraded":"2023-10-19T10:37:00.278315+0000","last_fullsized":"2023-10-19T10:37:00.278315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097588+0000","last_change":"2023-10-19T10:36:55.110137+0000","last_active":"2023-10-19T10:37:00.097588+0000","last_peered":"2023-10-19T10:37:00.097588+0000","last_clean":"2023-10-19T10:37:00.097588+0000","last_became_active":"2023-10-19T10:36:55.109003+0000","last_became_peered":"2023-10-19T10:36:55.109003+0000","last_unstale":"2023-10-19T10:37:00.097588+0000","last_undegraded":"2023-10-19T10:37:00.097588+0000","last_fullsized":"2023-10-19T10:37:00.097588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:38.218180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098351+0000","last_change":"2023-10-19T10:36:55.115362+0000","last_active":"2023-10-19T10:37:00.098351+0000","last_peered":"2023-10-19T10:37:00.098351+0000","last_clean":"2023-10-19T10:37:00.098351+0000","last_became_active":"2023-10-19T10:36:55.114806+0000","last_became_peered":"2023-10-19T10:36:55.114806+0000","last_unstale":"2023-10-19T10:37:00.098351+0000","last_undegraded":"2023-10-19T10:37:00.098351+0000","last_fullsized":"2023-10-19T10:37:00.098351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323850+0000","last_change":"2023-10-19T10:36:55.106573+0000","last_active":"2023-10-19T10:37:00.323850+0000","last_peered":"2023-10-19T10:37:00.323850+0000","last_clean":"2023-10-19T10:37:00.323850+0000","last_became_active":"2023-10-19T10:36:55.106293+0000","last_became_peered":"2023-10-19T10:36:55.106293+0000","last_unstale":"2023-10-19T10:37:00.323850+0000","last_undegraded":"2023-10-19T10:37:00.323850+0000","last_fullsized":"2023-10-19T10:37:00.323850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323255+0000","last_change":"2023-10-19T10:36:55.116400+0000","last_active":"2023-10-19T10:37:00.323255+0000","last_peered":"2023-10-19T10:37:00.323255+0000","last_clean":"2023-10-19T10:37:00.323255+0000","last_became_active":"2023-10-19T10:36:55.116142+0000","last_became_peered":"2023-10-19T10:36:55.116142+0000","last_unstale":"2023-10-19T10:37:00.323255+0000","last_undegraded":"2023-10-19T10:37:00.323255+0000","last_fullsized":"2023-10-19T10:37:00.323255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:00:11.139684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323751+0000","last_change":"2023-10-19T10:36:55.102152+0000","last_active":"2023-10-19T10:37:00.323751+0000","last_peered":"2023-10-19T10:37:00.323751+0000","last_clean":"2023-10-19T10:37:00.323751+0000","last_became_active":"2023-10-19T10:36:55.101878+0000","last_became_peered":"2023-10-19T10:36:55.101878+0000","last_unstale":"2023-10-19T10:37:00.323751+0000","last_undegraded":"2023-10-19T10:37:00.323751+0000","last_fullsized":"2023-10-19T10:37:00.323751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:27:50.253900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099451+0000","last_change":"2023-10-19T10:36:55.111165+0000","last_active":"2023-10-19T10:37:00.099451+0000","last_peered":"2023-10-19T10:37:00.099451+0000","last_clean":"2023-10-19T10:37:00.099451+0000","last_became_active":"2023-10-19T10:36:55.110828+0000","last_became_peered":"2023-10-19T10:36:55.110828+0000","last_unstale":"2023-10-19T10:37:00.099451+0000","last_undegraded":"2023-10-19T10:37:00.099451+0000","last_fullsized":"2023-10-19T10:37:00.099451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:19:39.989618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324638+0000","last_change":"2023-10-19T10:36:55.110928+0000","last_active":"2023-10-19T10:37:00.324638+0000","last_peered":"2023-10-19T10:37:00.324638+0000","last_clean":"2023-10-19T10:37:00.324638+0000","last_became_active":"2023-10-19T10:36:55.110647+0000","last_became_peered":"2023-10-19T10:36:55.110647+0000","last_unstale":"2023-10-19T10:37:00.324638+0000","last_undegraded":"2023-10-19T10:37:00.324638+0000","last_fullsized":"2023-10-19T10:37:00.324638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:17.439266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278188+0000","last_change":"2023-10-19T10:36:55.118334+0000","last_active":"2023-10-19T10:37:00.278188+0000","last_peered":"2023-10-19T10:37:00.278188+0000","last_clean":"2023-10-19T10:37:00.278188+0000","last_became_active":"2023-10-19T10:36:55.116957+0000","last_became_peered":"2023-10-19T10:36:55.116957+0000","last_unstale":"2023-10-19T10:37:00.278188+0000","last_undegraded":"2023-10-19T10:37:00.278188+0000","last_fullsized":"2023-10-19T10:37:00.278188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323040+0000","last_change":"2023-10-19T10:36:55.110747+0000","last_active":"2023-10-19T10:37:00.323040+0000","last_peered":"2023-10-19T10:37:00.323040+0000","last_clean":"2023-10-19T10:37:00.323040+0000","last_became_active":"2023-10-19T10:36:55.110453+0000","last_became_peered":"2023-10-19T10:36:55.110453+0000","last_unstale":"2023-10-19T10:37:00.323040+0000","last_undegraded":"2023-10-19T10:37:00.323040+0000","last_fullsized":"2023-10-19T10:37:00.323040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097980+0000","last_change":"2023-10-19T10:36:55.111146+0000","last_active":"2023-10-19T10:37:00.097980+0000","last_peered":"2023-10-19T10:37:00.097980+0000","last_clean":"2023-10-19T10:37:00.097980+0000","last_became_active":"2023-10-19T10:36:55.110894+0000","last_became_peered":"2023-10-19T10:36:55.110894+0000","last_unstale":"2023-10-19T10:37:00.097980+0000","last_undegraded":"2023-10-19T10:37:00.097980+0000","last_fullsized":"2023-10-19T10:37:00.097980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097815+0000","last_change":"2023-10-19T10:36:55.109335+0000","last_active":"2023-10-19T10:37:00.097815+0000","last_peered":"2023-10-19T10:37:00.097815+0000","last_clean":"2023-10-19T10:37:00.097815+0000","last_became_active":"2023-10-19T10:36:55.109002+0000","last_became_peered":"2023-10-19T10:36:55.109002+0000","last_unstale":"2023-10-19T10:37:00.097815+0000","last_undegraded":"2023-10-19T10:37:00.097815+0000","last_fullsized":"2023-10-19T10:37:00.097815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:32:10.062310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278344+0000","last_change":"2023-10-19T10:36:55.119014+0000","last_active":"2023-10-19T10:37:00.278344+0000","last_peered":"2023-10-19T10:37:00.278344+0000","last_clean":"2023-10-19T10:37:00.278344+0000","last_became_active":"2023-10-19T10:36:55.118470+0000","last_became_peered":"2023-10-19T10:36:55.118470+0000","last_unstale":"2023-10-19T10:37:00.278344+0000","last_undegraded":"2023-10-19T10:37:00.278344+0000","last_fullsized":"2023-10-19T10:37:00.278344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323295+0000","last_change":"2023-10-19T10:36:55.114470+0000","last_active":"2023-10-19T10:37:00.323295+0000","last_peered":"2023-10-19T10:37:00.323295+0000","last_clean":"2023-10-19T10:37:00.323295+0000","last_became_active":"2023-10-19T10:36:55.113809+0000","last_became_peered":"2023-10-19T10:36:55.113809+0000","last_unstale":"2023-10-19T10:37:00.323295+0000","last_undegraded":"2023-10-19T10:37:00.323295+0000","last_fullsized":"2023-10-19T10:37:00.323295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:22:52.211277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099367+0000","last_change":"2023-10-19T10:36:55.113901+0000","last_active":"2023-10-19T10:37:00.099367+0000","last_peered":"2023-10-19T10:37:00.099367+0000","last_clean":"2023-10-19T10:37:00.099367+0000","last_became_active":"2023-10-19T10:36:55.113629+0000","last_became_peered":"2023-10-19T10:36:55.113629+0000","last_unstale":"2023-10-19T10:37:00.099367+0000","last_undegraded":"2023-10-19T10:37:00.099367+0000","last_fullsized":"2023-10-19T10:37:00.099367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:20:21.320100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324574+0000","last_change":"2023-10-19T10:36:55.112724+0000","last_active":"2023-10-19T10:37:00.324574+0000","last_peered":"2023-10-19T10:37:00.324574+0000","last_clean":"2023-10-19T10:37:00.324574+0000","last_became_active":"2023-10-19T10:36:55.112307+0000","last_became_peered":"2023-10-19T10:36:55.112307+0000","last_unstale":"2023-10-19T10:37:00.324574+0000","last_undegraded":"2023-10-19T10:37:00.324574+0000","last_fullsized":"2023-10-19T10:37:00.324574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:51.008519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324311+0000","last_change":"2023-10-19T10:36:55.107659+0000","last_active":"2023-10-19T10:37:00.324311+0000","last_peered":"2023-10-19T10:37:00.324311+0000","last_clean":"2023-10-19T10:37:00.324311+0000","last_became_active":"2023-10-19T10:36:55.107382+0000","last_became_peered":"2023-10-19T10:36:55.107382+0000","last_unstale":"2023-10-19T10:37:00.324311+0000","last_undegraded":"2023-10-19T10:37:00.324311+0000","last_fullsized":"2023-10-19T10:37:00.324311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":294912,"data_stored":119711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":888832,"data_stored":710571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:37:24.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:37:24.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.948+0000 7fbfd5f18640 1 Processor -- start 2023-10-19T10:37:24.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.948+0000 7fbfd5f18640 1 -- start start 2023-10-19T10:37:24.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfd5f18640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbfd00d0e30 0x7fbfd015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfd5f18640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbfd00cd920 0x7fbfd00d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfd5f18640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd006aa30 con 0x7fbfd015c570 2023-10-19T10:37:24.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfd5f18640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd015ce90 con 0x7fbfd00d0e30 2023-10-19T10:37:24.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfd5f18640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd006b2d0 con 0x7fbfd00cd920 2023-10-19T10:37:24.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbfd00d0e30 0x7fbfd015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbfd00d0e30 0x7fbfd015c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39264/0 (socket says 172.21.15.50:39264) 2023-10-19T10:37:24.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 -- 172.21.15.50:0/2781824354 learned_addr learned my addr 172.21.15.50:0/2781824354 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:24.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfcffff640 1 --2- 172.21.15.50:0/2781824354 >> v2:172.21.15.183:3300/0 conn(0x7fbfd00cd920 0x7fbfd00d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 -- 172.21.15.50:0/2781824354 >> v2:172.21.15.183:3300/0 conn(0x7fbfd00cd920 msgr2=0x7fbfd00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:24.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 --2- 172.21.15.50:0/2781824354 >> v2:172.21.15.183:3300/0 conn(0x7fbfd00cd920 0x7fbfd00d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 -- 172.21.15.50:0/2781824354 >> v1:172.21.15.183:6789/0 conn(0x7fbfd015c570 legacy=0x7fbfd006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfceffd640 1 -- 172.21.15.50:0/2781824354 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00d1270 con 0x7fbfd00d0e30 2023-10-19T10:37:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.949+0000 7fbfcffff640 1 --2- 172.21.15.50:0/2781824354 >> v2:172.21.15.183:3300/0 conn(0x7fbfd00cd920 0x7fbfd00d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfceffd640 1 --2- 172.21.15.50:0/2781824354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd00d0e30 0x7fbfd015c950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00d3770 tx=0x7fbfc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e46a88a48acee6d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfce7fc640 1 -- 172.21.15.50:0/2781824354 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4031870 con 0x7fbfd00d0e30 2023-10-19T10:37:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfce7fc640 1 -- 172.21.15.50:0/2781824354 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc40319d0 con 0x7fbfd00d0e30 2023-10-19T10:37:24.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfce7fc640 1 -- 172.21.15.50:0/2781824354 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4031d60 con 0x7fbfd00d0e30 2023-10-19T10:37:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfd5f18640 1 -- 172.21.15.50:0/2781824354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd00d0e30 msgr2=0x7fbfd015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/2781824354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd00d0e30 0x7fbfd015c950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00d3770 tx=0x7fbfc402b910 comp rx=0 tx=0).stop 2023-10-19T10:37:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfd5f18640 1 -- 172.21.15.50:0/2781824354 shutdown_connections 2023-10-19T10:37:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.950+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/2781824354 >> v2:172.21.15.183:3300/0 conn(0x7fbfd00cd920 0x7fbfd00d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.951+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/2781824354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd00d0e30 0x7fbfd015c950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.951+0000 7fbfd5f18640 1 -- 172.21.15.50:0/2781824354 >> 172.21.15.50:0/2781824354 conn(0x7fbfd0089ff0 msgr2=0x7fbfd009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.951+0000 7fbfd5f18640 1 -- 172.21.15.50:0/2781824354 shutdown_connections 2023-10-19T10:37:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.951+0000 7fbfd5f18640 1 -- 172.21.15.50:0/2781824354 wait complete. 2023-10-19T10:37:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.952+0000 7fbfd5f18640 1 Processor -- start 2023-10-19T10:37:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.952+0000 7fbfd5f18640 1 -- start start 2023-10-19T10:37:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfd00cd920 0x7fbfd01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfd00d0e30 0x7fbfd00c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 0x7fbfd00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd015dd50 con 0x7fbfd015c570 2023-10-19T10:37:24.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd015dff0 con 0x7fbfd00cd920 2023-10-19T10:37:24.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd015e2d0 con 0x7fbfd00d0e30 2023-10-19T10:37:24.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 0x7fbfd00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 0x7fbfd00c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39266/0 (socket says 172.21.15.50:39266) 2023-10-19T10:37:24.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 -- 172.21.15.50:0/3419975120 learned_addr learned my addr 172.21.15.50:0/3419975120 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:24.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfceffd640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfd00d0e30 0x7fbfd00c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcf7fe640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfd00cd920 0x7fbfd01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 -- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfd00d0e30 msgr2=0x7fbfd00c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfd00d0e30 0x7fbfd00c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 -- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfd00cd920 msgr2=0x7fbfd01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:24.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfd00cd920 0x7fbfd01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:24.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 -- 172.21.15.50:0/3419975120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40094d0 con 0x7fbfd015c570 2023-10-19T10:37:24.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfcffff640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 0x7fbfd00c7590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8007d40 tx=0x7fbfc800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8003990 con 0x7fbfd015c570 2023-10-19T10:37:24.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc800f040 con 0x7fbfd015c570 2023-10-19T10:37:24.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd015e5b0 con 0x7fbfd015c570 2023-10-19T10:37:24.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8003d30 con 0x7fbfd015c570 2023-10-19T10:37:24.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.953+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd00c7ad0 con 0x7fbfd015c570 2023-10-19T10:37:24.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.954+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd015e7f0 con 0x7fbfd015c570 2023-10-19T10:37:24.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.955+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbfc8004030 con 0x7fbfd015c570 2023-10-19T10:37:24.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.955+0000 7fbfb7fff640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbfbc06a050 0x7fbfbc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:24.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.955+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbfc80889c0 con 0x7fbfd015c570 2023-10-19T10:37:24.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.955+0000 7fbfcf7fe640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbfbc06a050 0x7fbfbc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:24.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.956+0000 7fbfcf7fe640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbfbc06a050 0x7fbfbc06c510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0007fd0 tx=0x7fbfc0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:24.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:24.959+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbfd015e7f0 con 0x7fbfd015c570 2023-10-19T10:37:25.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.132+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfd015e160 con 0x7fbfbc06a050 2023-10-19T10:37:25.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.142+0000 7fbfb7fff640 1 -- 172.21.15.50:0/3419975120 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367360 (secure 0 0 0) 0x7fbfd015e160 con 0x7fbfbc06a050 2023-10-19T10:37:25.148 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:25.150 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:37:25.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbfbc06a050 msgr2=0x7fbfbc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbfbc06a050 0x7fbfbc06c510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0007fd0 tx=0x7fbfc0009040 comp rx=0 tx=0).stop 2023-10-19T10:37:25.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 msgr2=0x7fbfd00c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 0x7fbfd00c7590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8007d40 tx=0x7fbfc800d4a0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 shutdown_connections 2023-10-19T10:37:25.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbfbc06a050 0x7fbfbc06c510 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfd015c570 0x7fbfd00c7590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfd00d0e30 0x7fbfd00c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 --2- 172.21.15.50:0/3419975120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfd00cd920 0x7fbfd01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 >> 172.21.15.50:0/3419975120 conn(0x7fbfd0089ff0 msgr2=0x7fbfd00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:25.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 shutdown_connections 2023-10-19T10:37:25.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.145+0000 7fbfd5f18640 1 -- 172.21.15.50:0/3419975120 wait complete. 2023-10-19T10:37:25.159 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-10-19T10:37:24.470680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236080,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738640,"statfs":{"total":773094113280,"available":772852367360,"internally_reserved":0,"allocated":3579904,"data_stored":2146982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003722"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096693+0000","last_change":"2023-10-19T10:36:59.359352+0000","last_active":"2023-10-19T10:37:00.096693+0000","last_peered":"2023-10-19T10:37:00.096693+0000","last_clean":"2023-10-19T10:37:00.096693+0000","last_became_active":"2023-10-19T10:36:59.358992+0000","last_became_peered":"2023-10-19T10:36:59.358992+0000","last_unstale":"2023-10-19T10:37:00.096693+0000","last_undegraded":"2023-10-19T10:37:00.096693+0000","last_fullsized":"2023-10-19T10:37:00.096693+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:36:39.869930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278383+0000","last_change":"2023-10-19T10:36:54.305740+0000","last_active":"2023-10-19T10:37:00.278383+0000","last_peered":"2023-10-19T10:37:00.278383+0000","last_clean":"2023-10-19T10:37:00.278383+0000","last_became_active":"2023-10-19T10:36:54.305192+0000","last_became_peered":"2023-10-19T10:36:54.305192+0000","last_unstale":"2023-10-19T10:37:00.278383+0000","last_undegraded":"2023-10-19T10:37:00.278383+0000","last_fullsized":"2023-10-19T10:37:00.278383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097666+0000","last_change":"2023-10-19T10:36:55.108193+0000","last_active":"2023-10-19T10:37:00.097666+0000","last_peered":"2023-10-19T10:37:00.097666+0000","last_clean":"2023-10-19T10:37:00.097666+0000","last_became_active":"2023-10-19T10:36:55.107375+0000","last_became_peered":"2023-10-19T10:36:55.107375+0000","last_unstale":"2023-10-19T10:37:00.097666+0000","last_undegraded":"2023-10-19T10:37:00.097666+0000","last_fullsized":"2023-10-19T10:37:00.097666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:13:03.441540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097567+0000","last_change":"2023-10-19T10:36:59.344962+0000","last_active":"2023-10-19T10:37:00.097567+0000","last_peered":"2023-10-19T10:37:00.097567+0000","last_clean":"2023-10-19T10:37:00.097567+0000","last_became_active":"2023-10-19T10:36:59.344519+0000","last_became_peered":"2023-10-19T10:36:59.344519+0000","last_unstale":"2023-10-19T10:37:00.097567+0000","last_undegraded":"2023-10-19T10:37:00.097567+0000","last_fullsized":"2023-10-19T10:37:00.097567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:33.660694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:54.310896+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:54.310583+0000","last_became_peered":"2023-10-19T10:36:54.310583+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:56:34.283482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323668+0000","last_change":"2023-10-19T10:36:55.104115+0000","last_active":"2023-10-19T10:37:00.323668+0000","last_peered":"2023-10-19T10:37:00.323668+0000","last_clean":"2023-10-19T10:37:00.323668+0000","last_became_active":"2023-10-19T10:36:55.103785+0000","last_became_peered":"2023-10-19T10:36:55.103785+0000","last_unstale":"2023-10-19T10:37:00.323668+0000","last_undegraded":"2023-10-19T10:37:00.323668+0000","last_fullsized":"2023-10-19T10:37:00.323668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324072+0000","last_change":"2023-10-19T10:36:59.350493+0000","last_active":"2023-10-19T10:37:00.324072+0000","last_peered":"2023-10-19T10:37:00.324072+0000","last_clean":"2023-10-19T10:37:00.324072+0000","last_became_active":"2023-10-19T10:36:59.350100+0000","last_became_peered":"2023-10-19T10:36:59.350100+0000","last_unstale":"2023-10-19T10:37:00.324072+0000","last_undegraded":"2023-10-19T10:37:00.324072+0000","last_fullsized":"2023-10-19T10:37:00.324072+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324355+0000","last_change":"2023-10-19T10:36:54.307087+0000","last_active":"2023-10-19T10:37:00.324355+0000","last_peered":"2023-10-19T10:37:00.324355+0000","last_clean":"2023-10-19T10:37:00.324355+0000","last_became_active":"2023-10-19T10:36:54.306818+0000","last_became_peered":"2023-10-19T10:36:54.306818+0000","last_unstale":"2023-10-19T10:37:00.324355+0000","last_undegraded":"2023-10-19T10:37:00.324355+0000","last_fullsized":"2023-10-19T10:37:00.324355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:25.504656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098032+0000","last_change":"2023-10-19T10:36:55.111676+0000","last_active":"2023-10-19T10:37:00.098032+0000","last_peered":"2023-10-19T10:37:00.098032+0000","last_clean":"2023-10-19T10:37:00.098032+0000","last_became_active":"2023-10-19T10:36:55.110442+0000","last_became_peered":"2023-10-19T10:36:55.110442+0000","last_unstale":"2023-10-19T10:37:00.098032+0000","last_undegraded":"2023-10-19T10:37:00.098032+0000","last_fullsized":"2023-10-19T10:37:00.098032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:51:02.897898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097851+0000","last_change":"2023-10-19T10:36:59.357016+0000","last_active":"2023-10-19T10:37:00.097851+0000","last_peered":"2023-10-19T10:37:00.097851+0000","last_clean":"2023-10-19T10:37:00.097851+0000","last_became_active":"2023-10-19T10:36:59.356666+0000","last_became_peered":"2023-10-19T10:36:59.356666+0000","last_unstale":"2023-10-19T10:37:00.097851+0000","last_undegraded":"2023-10-19T10:37:00.097851+0000","last_fullsized":"2023-10-19T10:37:00.097851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324054+0000","last_change":"2023-10-19T10:36:54.306526+0000","last_active":"2023-10-19T10:37:00.324054+0000","last_peered":"2023-10-19T10:37:00.324054+0000","last_clean":"2023-10-19T10:37:00.324054+0000","last_became_active":"2023-10-19T10:36:54.306285+0000","last_became_peered":"2023-10-19T10:36:54.306285+0000","last_unstale":"2023-10-19T10:37:00.324054+0000","last_undegraded":"2023-10-19T10:37:00.324054+0000","last_fullsized":"2023-10-19T10:37:00.324054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:55.717306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098946+0000","last_change":"2023-10-19T10:36:55.115029+0000","last_active":"2023-10-19T10:37:00.098946+0000","last_peered":"2023-10-19T10:37:00.098946+0000","last_clean":"2023-10-19T10:37:00.098946+0000","last_became_active":"2023-10-19T10:36:55.114754+0000","last_became_peered":"2023-10-19T10:36:55.114754+0000","last_unstale":"2023-10-19T10:37:00.098946+0000","last_undegraded":"2023-10-19T10:37:00.098946+0000","last_fullsized":"2023-10-19T10:37:00.098946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:30:31.789571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097648+0000","last_change":"2023-10-19T10:36:59.355687+0000","last_active":"2023-10-19T10:37:00.097648+0000","last_peered":"2023-10-19T10:37:00.097648+0000","last_clean":"2023-10-19T10:37:00.097648+0000","last_became_active":"2023-10-19T10:36:59.355294+0000","last_became_peered":"2023-10-19T10:36:59.355294+0000","last_unstale":"2023-10-19T10:37:00.097648+0000","last_undegraded":"2023-10-19T10:37:00.097648+0000","last_fullsized":"2023-10-19T10:37:00.097648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:39:00.759461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278352+0000","last_change":"2023-10-19T10:36:54.307829+0000","last_active":"2023-10-19T10:37:00.278352+0000","last_peered":"2023-10-19T10:37:00.278352+0000","last_clean":"2023-10-19T10:37:00.278352+0000","last_became_active":"2023-10-19T10:36:54.307571+0000","last_became_peered":"2023-10-19T10:36:54.307571+0000","last_unstale":"2023-10-19T10:37:00.278352+0000","last_undegraded":"2023-10-19T10:37:00.278352+0000","last_fullsized":"2023-10-19T10:37:00.278352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:45.887676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323888+0000","last_change":"2023-10-19T10:36:55.112212+0000","last_active":"2023-10-19T10:37:00.323888+0000","last_peered":"2023-10-19T10:37:00.323888+0000","last_clean":"2023-10-19T10:37:00.323888+0000","last_became_active":"2023-10-19T10:36:55.111854+0000","last_became_peered":"2023-10-19T10:36:55.111854+0000","last_unstale":"2023-10-19T10:37:00.323888+0000","last_undegraded":"2023-10-19T10:37:00.323888+0000","last_fullsized":"2023-10-19T10:37:00.323888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:49:46.713742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099140+0000","last_change":"2023-10-19T10:36:59.352206+0000","last_active":"2023-10-19T10:37:00.099140+0000","last_peered":"2023-10-19T10:37:00.099140+0000","last_clean":"2023-10-19T10:37:00.099140+0000","last_became_active":"2023-10-19T10:36:59.351813+0000","last_became_peered":"2023-10-19T10:36:59.351813+0000","last_unstale":"2023-10-19T10:37:00.099140+0000","last_undegraded":"2023-10-19T10:37:00.099140+0000","last_fullsized":"2023-10-19T10:37:00.099140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:02.738858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098273+0000","last_change":"2023-10-19T10:36:54.305390+0000","last_active":"2023-10-19T10:37:00.098273+0000","last_peered":"2023-10-19T10:37:00.098273+0000","last_clean":"2023-10-19T10:37:00.098273+0000","last_became_active":"2023-10-19T10:36:54.304960+0000","last_became_peered":"2023-10-19T10:36:54.304960+0000","last_unstale":"2023-10-19T10:37:00.098273+0000","last_undegraded":"2023-10-19T10:37:00.098273+0000","last_fullsized":"2023-10-19T10:37:00.098273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:21.713184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324108+0000","last_change":"2023-10-19T10:36:55.114266+0000","last_active":"2023-10-19T10:37:00.324108+0000","last_peered":"2023-10-19T10:37:00.324108+0000","last_clean":"2023-10-19T10:37:00.324108+0000","last_became_active":"2023-10-19T10:36:55.113992+0000","last_became_peered":"2023-10-19T10:36:55.113992+0000","last_unstale":"2023-10-19T10:37:00.324108+0000","last_undegraded":"2023-10-19T10:37:00.324108+0000","last_fullsized":"2023-10-19T10:37:00.324108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:17.080140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323688+0000","last_change":"2023-10-19T10:36:59.349278+0000","last_active":"2023-10-19T10:37:00.323688+0000","last_peered":"2023-10-19T10:37:00.323688+0000","last_clean":"2023-10-19T10:37:00.323688+0000","last_became_active":"2023-10-19T10:36:59.348852+0000","last_became_peered":"2023-10-19T10:36:59.348852+0000","last_unstale":"2023-10-19T10:37:00.323688+0000","last_undegraded":"2023-10-19T10:37:00.323688+0000","last_fullsized":"2023-10-19T10:37:00.323688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:20.365692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096958+0000","last_change":"2023-10-19T10:36:54.306039+0000","last_active":"2023-10-19T10:37:00.096958+0000","last_peered":"2023-10-19T10:37:00.096958+0000","last_clean":"2023-10-19T10:37:00.096958+0000","last_became_active":"2023-10-19T10:36:54.304502+0000","last_became_peered":"2023-10-19T10:36:54.304502+0000","last_unstale":"2023-10-19T10:37:00.096958+0000","last_undegraded":"2023-10-19T10:37:00.096958+0000","last_fullsized":"2023-10-19T10:37:00.096958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:07:52.900648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278255+0000","last_change":"2023-10-19T10:36:55.114034+0000","last_active":"2023-10-19T10:37:00.278255+0000","last_peered":"2023-10-19T10:37:00.278255+0000","last_clean":"2023-10-19T10:37:00.278255+0000","last_became_active":"2023-10-19T10:36:55.113734+0000","last_became_peered":"2023-10-19T10:36:55.113734+0000","last_unstale":"2023-10-19T10:37:00.278255+0000","last_undegraded":"2023-10-19T10:37:00.278255+0000","last_fullsized":"2023-10-19T10:37:00.278255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278113+0000","last_change":"2023-10-19T10:36:59.349586+0000","last_active":"2023-10-19T10:37:00.278113+0000","last_peered":"2023-10-19T10:37:00.278113+0000","last_clean":"2023-10-19T10:37:00.278113+0000","last_became_active":"2023-10-19T10:36:59.349176+0000","last_became_peered":"2023-10-19T10:36:59.349176+0000","last_unstale":"2023-10-19T10:37:00.278113+0000","last_undegraded":"2023-10-19T10:37:00.278113+0000","last_fullsized":"2023-10-19T10:37:00.278113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323262+0000","last_change":"2023-10-19T10:36:54.310735+0000","last_active":"2023-10-19T10:37:00.323262+0000","last_peered":"2023-10-19T10:37:00.323262+0000","last_clean":"2023-10-19T10:37:00.323262+0000","last_became_active":"2023-10-19T10:36:54.309734+0000","last_became_peered":"2023-10-19T10:36:54.309734+0000","last_unstale":"2023-10-19T10:37:00.323262+0000","last_undegraded":"2023-10-19T10:37:00.323262+0000","last_fullsized":"2023-10-19T10:37:00.323262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323712+0000","last_change":"2023-10-19T10:36:55.113078+0000","last_active":"2023-10-19T10:37:00.323712+0000","last_peered":"2023-10-19T10:37:00.323712+0000","last_clean":"2023-10-19T10:37:00.323712+0000","last_became_active":"2023-10-19T10:36:55.112775+0000","last_became_peered":"2023-10-19T10:36:55.112775+0000","last_unstale":"2023-10-19T10:37:00.323712+0000","last_undegraded":"2023-10-19T10:37:00.323712+0000","last_fullsized":"2023-10-19T10:37:00.323712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:12:41.211214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322933+0000","last_change":"2023-10-19T10:36:55.105391+0000","last_active":"2023-10-19T10:37:00.322933+0000","last_peered":"2023-10-19T10:37:00.322933+0000","last_clean":"2023-10-19T10:37:00.322933+0000","last_became_active":"2023-10-19T10:36:55.105112+0000","last_became_peered":"2023-10-19T10:36:55.105112+0000","last_unstale":"2023-10-19T10:37:00.322933+0000","last_undegraded":"2023-10-19T10:37:00.322933+0000","last_fullsized":"2023-10-19T10:37:00.322933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278476+0000","last_change":"2023-10-19T10:36:54.305435+0000","last_active":"2023-10-19T10:37:00.278476+0000","last_peered":"2023-10-19T10:37:00.278476+0000","last_clean":"2023-10-19T10:37:00.278476+0000","last_became_active":"2023-10-19T10:36:54.305123+0000","last_became_peered":"2023-10-19T10:36:54.305123+0000","last_unstale":"2023-10-19T10:37:00.278476+0000","last_undegraded":"2023-10-19T10:37:00.278476+0000","last_fullsized":"2023-10-19T10:37:00.278476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323209+0000","last_change":"2023-10-19T10:36:59.350259+0000","last_active":"2023-10-19T10:37:00.323209+0000","last_peered":"2023-10-19T10:37:00.323209+0000","last_clean":"2023-10-19T10:37:00.323209+0000","last_became_active":"2023-10-19T10:36:59.349878+0000","last_became_peered":"2023-10-19T10:36:59.349878+0000","last_unstale":"2023-10-19T10:37:00.323209+0000","last_undegraded":"2023-10-19T10:37:00.323209+0000","last_fullsized":"2023-10-19T10:37:00.323209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323250+0000","last_change":"2023-10-19T10:36:55.113112+0000","last_active":"2023-10-19T10:37:00.323250+0000","last_peered":"2023-10-19T10:37:00.323250+0000","last_clean":"2023-10-19T10:37:00.323250+0000","last_became_active":"2023-10-19T10:36:55.110232+0000","last_became_peered":"2023-10-19T10:36:55.110232+0000","last_unstale":"2023-10-19T10:37:00.323250+0000","last_undegraded":"2023-10-19T10:37:00.323250+0000","last_fullsized":"2023-10-19T10:37:00.323250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324722+0000","last_change":"2023-10-19T10:36:54.310064+0000","last_active":"2023-10-19T10:37:00.324722+0000","last_peered":"2023-10-19T10:37:00.324722+0000","last_clean":"2023-10-19T10:37:00.324722+0000","last_became_active":"2023-10-19T10:36:54.309581+0000","last_became_peered":"2023-10-19T10:36:54.309581+0000","last_unstale":"2023-10-19T10:37:00.324722+0000","last_undegraded":"2023-10-19T10:37:00.324722+0000","last_fullsized":"2023-10-19T10:37:00.324722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:58:16.076243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098023+0000","last_change":"2023-10-19T10:36:59.355292+0000","last_active":"2023-10-19T10:37:00.098023+0000","last_peered":"2023-10-19T10:37:00.098023+0000","last_clean":"2023-10-19T10:37:00.098023+0000","last_became_active":"2023-10-19T10:36:59.354866+0000","last_became_peered":"2023-10-19T10:36:59.354866+0000","last_unstale":"2023-10-19T10:37:00.098023+0000","last_undegraded":"2023-10-19T10:37:00.098023+0000","last_fullsized":"2023-10-19T10:37:00.098023+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098814+0000","last_change":"2023-10-19T10:36:54.310722+0000","last_active":"2023-10-19T10:37:00.098814+0000","last_peered":"2023-10-19T10:37:00.098814+0000","last_clean":"2023-10-19T10:37:00.098814+0000","last_became_active":"2023-10-19T10:36:54.310410+0000","last_became_peered":"2023-10-19T10:36:54.310410+0000","last_unstale":"2023-10-19T10:37:00.098814+0000","last_undegraded":"2023-10-19T10:37:00.098814+0000","last_fullsized":"2023-10-19T10:37:00.098814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:00.221492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097163+0000","last_change":"2023-10-19T10:36:55.110762+0000","last_active":"2023-10-19T10:37:00.097163+0000","last_peered":"2023-10-19T10:37:00.097163+0000","last_clean":"2023-10-19T10:37:00.097163+0000","last_became_active":"2023-10-19T10:36:55.110503+0000","last_became_peered":"2023-10-19T10:36:55.110503+0000","last_unstale":"2023-10-19T10:37:00.097163+0000","last_undegraded":"2023-10-19T10:37:00.097163+0000","last_fullsized":"2023-10-19T10:37:00.097163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:14.103808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323380+0000","last_change":"2023-10-19T10:36:59.352779+0000","last_active":"2023-10-19T10:37:00.323380+0000","last_peered":"2023-10-19T10:37:00.323380+0000","last_clean":"2023-10-19T10:37:00.323380+0000","last_became_active":"2023-10-19T10:36:59.352362+0000","last_became_peered":"2023-10-19T10:36:59.352362+0000","last_unstale":"2023-10-19T10:37:00.323380+0000","last_undegraded":"2023-10-19T10:37:00.323380+0000","last_fullsized":"2023-10-19T10:37:00.323380+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:53.134651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:04.746420+0000","last_change":"2023-10-19T10:36:54.310368+0000","last_active":"2023-10-19T10:37:04.746420+0000","last_peered":"2023-10-19T10:37:04.746420+0000","last_clean":"2023-10-19T10:37:04.746420+0000","last_became_active":"2023-10-19T10:36:54.309745+0000","last_became_peered":"2023-10-19T10:36:54.309745+0000","last_unstale":"2023-10-19T10:37:04.746420+0000","last_undegraded":"2023-10-19T10:37:04.746420+0000","last_fullsized":"2023-10-19T10:37:04.746420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:15:27.898150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099416+0000","last_change":"2023-10-19T10:36:55.106300+0000","last_active":"2023-10-19T10:37:00.099416+0000","last_peered":"2023-10-19T10:37:00.099416+0000","last_clean":"2023-10-19T10:37:00.099416+0000","last_became_active":"2023-10-19T10:36:55.106045+0000","last_became_peered":"2023-10-19T10:36:55.106045+0000","last_unstale":"2023-10-19T10:37:00.099416+0000","last_undegraded":"2023-10-19T10:37:00.099416+0000","last_fullsized":"2023-10-19T10:37:00.099416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:28:29.050287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096752+0000","last_change":"2023-10-19T10:36:59.358801+0000","last_active":"2023-10-19T10:37:00.096752+0000","last_peered":"2023-10-19T10:37:00.096752+0000","last_clean":"2023-10-19T10:37:00.096752+0000","last_became_active":"2023-10-19T10:36:59.356906+0000","last_became_peered":"2023-10-19T10:36:59.356906+0000","last_unstale":"2023-10-19T10:37:00.096752+0000","last_undegraded":"2023-10-19T10:37:00.096752+0000","last_fullsized":"2023-10-19T10:37:00.096752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:56.099256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097964+0000","last_change":"2023-10-19T10:36:55.105397+0000","last_active":"2023-10-19T10:37:00.097964+0000","last_peered":"2023-10-19T10:37:00.097964+0000","last_clean":"2023-10-19T10:37:00.097964+0000","last_became_active":"2023-10-19T10:36:55.105016+0000","last_became_peered":"2023-10-19T10:36:55.105016+0000","last_unstale":"2023-10-19T10:37:00.097964+0000","last_undegraded":"2023-10-19T10:37:00.097964+0000","last_fullsized":"2023-10-19T10:37:00.097964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:13.604819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323795+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:37:00.323795+0000","last_peered":"2023-10-19T10:37:00.323795+0000","last_clean":"2023-10-19T10:37:00.323795+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:37:00.323795+0000","last_undegraded":"2023-10-19T10:37:00.323795+0000","last_fullsized":"2023-10-19T10:37:00.323795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:47:38.405947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278495+0000","last_change":"2023-10-19T10:36:54.301501+0000","last_active":"2023-10-19T10:37:00.278495+0000","last_peered":"2023-10-19T10:37:00.278495+0000","last_clean":"2023-10-19T10:37:00.278495+0000","last_became_active":"2023-10-19T10:36:54.301193+0000","last_became_peered":"2023-10-19T10:36:54.301193+0000","last_unstale":"2023-10-19T10:37:00.278495+0000","last_undegraded":"2023-10-19T10:37:00.278495+0000","last_fullsized":"2023-10-19T10:37:00.278495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323709+0000","last_change":"2023-10-19T10:36:59.351528+0000","last_active":"2023-10-19T10:37:00.323709+0000","last_peered":"2023-10-19T10:37:00.323709+0000","last_clean":"2023-10-19T10:37:00.323709+0000","last_became_active":"2023-10-19T10:36:59.351102+0000","last_became_peered":"2023-10-19T10:36:59.351102+0000","last_unstale":"2023-10-19T10:37:00.323709+0000","last_undegraded":"2023-10-19T10:37:00.323709+0000","last_fullsized":"2023-10-19T10:37:00.323709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323080+0000","last_change":"2023-10-19T10:36:55.108015+0000","last_active":"2023-10-19T10:37:00.323080+0000","last_peered":"2023-10-19T10:37:00.323080+0000","last_clean":"2023-10-19T10:37:00.323080+0000","last_became_active":"2023-10-19T10:36:55.107693+0000","last_became_peered":"2023-10-19T10:36:55.107693+0000","last_unstale":"2023-10-19T10:37:00.323080+0000","last_undegraded":"2023-10-19T10:37:00.323080+0000","last_fullsized":"2023-10-19T10:37:00.323080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097888+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:37:00.097888+0000","last_peered":"2023-10-19T10:37:00.097888+0000","last_clean":"2023-10-19T10:37:00.097888+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:37:00.097888+0000","last_undegraded":"2023-10-19T10:37:00.097888+0000","last_fullsized":"2023-10-19T10:37:00.097888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:31:56.734648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324508+0000","last_change":"2023-10-19T10:36:54.310814+0000","last_active":"2023-10-19T10:37:00.324508+0000","last_peered":"2023-10-19T10:37:00.324508+0000","last_clean":"2023-10-19T10:37:00.324508+0000","last_became_active":"2023-10-19T10:36:54.310517+0000","last_became_peered":"2023-10-19T10:36:54.310517+0000","last_unstale":"2023-10-19T10:37:00.324508+0000","last_undegraded":"2023-10-19T10:37:00.324508+0000","last_fullsized":"2023-10-19T10:37:00.324508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:37.214283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097777+0000","last_change":"2023-10-19T10:36:59.353687+0000","last_active":"2023-10-19T10:37:00.097777+0000","last_peered":"2023-10-19T10:37:00.097777+0000","last_clean":"2023-10-19T10:37:00.097777+0000","last_became_active":"2023-10-19T10:36:59.353297+0000","last_became_peered":"2023-10-19T10:36:59.353297+0000","last_unstale":"2023-10-19T10:37:00.097777+0000","last_undegraded":"2023-10-19T10:37:00.097777+0000","last_fullsized":"2023-10-19T10:37:00.097777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:05:36.096009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278152+0000","last_change":"2023-10-19T10:36:55.118741+0000","last_active":"2023-10-19T10:37:00.278152+0000","last_peered":"2023-10-19T10:37:00.278152+0000","last_clean":"2023-10-19T10:37:00.278152+0000","last_became_active":"2023-10-19T10:36:55.116465+0000","last_became_peered":"2023-10-19T10:36:55.116465+0000","last_unstale":"2023-10-19T10:37:00.278152+0000","last_undegraded":"2023-10-19T10:37:00.278152+0000","last_fullsized":"2023-10-19T10:37:00.278152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099045+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:37:00.099045+0000","last_peered":"2023-10-19T10:37:00.099045+0000","last_clean":"2023-10-19T10:37:00.099045+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:37:00.099045+0000","last_undegraded":"2023-10-19T10:37:00.099045+0000","last_fullsized":"2023-10-19T10:37:00.099045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:12.307342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097428+0000","last_change":"2023-10-19T10:36:54.306800+0000","last_active":"2023-10-19T10:37:00.097428+0000","last_peered":"2023-10-19T10:37:00.097428+0000","last_clean":"2023-10-19T10:37:00.097428+0000","last_became_active":"2023-10-19T10:36:54.306507+0000","last_became_peered":"2023-10-19T10:36:54.306507+0000","last_unstale":"2023-10-19T10:37:00.097428+0000","last_undegraded":"2023-10-19T10:37:00.097428+0000","last_fullsized":"2023-10-19T10:37:00.097428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:41.200123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323296+0000","last_change":"2023-10-19T10:36:59.352359+0000","last_active":"2023-10-19T10:37:00.323296+0000","last_peered":"2023-10-19T10:37:00.323296+0000","last_clean":"2023-10-19T10:37:00.323296+0000","last_became_active":"2023-10-19T10:36:59.351975+0000","last_became_peered":"2023-10-19T10:36:59.351975+0000","last_unstale":"2023-10-19T10:37:00.323296+0000","last_undegraded":"2023-10-19T10:37:00.323296+0000","last_fullsized":"2023-10-19T10:37:00.323296+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:39:36.789471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097181+0000","last_change":"2023-10-19T10:36:55.109753+0000","last_active":"2023-10-19T10:37:00.097181+0000","last_peered":"2023-10-19T10:37:00.097181+0000","last_clean":"2023-10-19T10:37:00.097181+0000","last_became_active":"2023-10-19T10:36:55.109424+0000","last_became_peered":"2023-10-19T10:36:55.109424+0000","last_unstale":"2023-10-19T10:37:00.097181+0000","last_undegraded":"2023-10-19T10:37:00.097181+0000","last_fullsized":"2023-10-19T10:37:00.097181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097504+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:37:00.097504+0000","last_peered":"2023-10-19T10:37:00.097504+0000","last_clean":"2023-10-19T10:37:00.097504+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:37:00.097504+0000","last_undegraded":"2023-10-19T10:37:00.097504+0000","last_fullsized":"2023-10-19T10:37:00.097504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:43.871421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324473+0000","last_change":"2023-10-19T10:36:54.305974+0000","last_active":"2023-10-19T10:37:00.324473+0000","last_peered":"2023-10-19T10:37:00.324473+0000","last_clean":"2023-10-19T10:37:00.324473+0000","last_became_active":"2023-10-19T10:36:54.305710+0000","last_became_peered":"2023-10-19T10:36:54.305710+0000","last_unstale":"2023-10-19T10:37:00.324473+0000","last_undegraded":"2023-10-19T10:37:00.324473+0000","last_fullsized":"2023-10-19T10:37:00.324473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277916+0000","last_change":"2023-10-19T10:36:59.353503+0000","last_active":"2023-10-19T10:37:00.277916+0000","last_peered":"2023-10-19T10:37:00.277916+0000","last_clean":"2023-10-19T10:37:00.277916+0000","last_became_active":"2023-10-19T10:36:59.352985+0000","last_became_peered":"2023-10-19T10:36:59.352985+0000","last_unstale":"2023-10-19T10:37:00.277916+0000","last_undegraded":"2023-10-19T10:37:00.277916+0000","last_fullsized":"2023-10-19T10:37:00.277916+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:07.395332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097415+0000","last_change":"2023-10-19T10:36:55.112577+0000","last_active":"2023-10-19T10:37:00.097415+0000","last_peered":"2023-10-19T10:37:00.097415+0000","last_clean":"2023-10-19T10:37:00.097415+0000","last_became_active":"2023-10-19T10:36:55.112302+0000","last_became_peered":"2023-10-19T10:36:55.112302+0000","last_unstale":"2023-10-19T10:37:00.097415+0000","last_undegraded":"2023-10-19T10:37:00.097415+0000","last_fullsized":"2023-10-19T10:37:00.097415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099330+0000","last_change":"2023-10-19T10:36:54.302990+0000","last_active":"2023-10-19T10:37:00.099330+0000","last_peered":"2023-10-19T10:37:00.099330+0000","last_clean":"2023-10-19T10:37:00.099330+0000","last_became_active":"2023-10-19T10:36:54.302553+0000","last_became_peered":"2023-10-19T10:36:54.302553+0000","last_unstale":"2023-10-19T10:37:00.099330+0000","last_undegraded":"2023-10-19T10:37:00.099330+0000","last_fullsized":"2023-10-19T10:37:00.099330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:07:06.761773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323861+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:37:00.323861+0000","last_peered":"2023-10-19T10:37:00.323861+0000","last_clean":"2023-10-19T10:37:00.323861+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:37:00.323861+0000","last_undegraded":"2023-10-19T10:37:00.323861+0000","last_fullsized":"2023-10-19T10:37:00.323861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:37.072096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277993+0000","last_change":"2023-10-19T10:36:59.351199+0000","last_active":"2023-10-19T10:37:00.277993+0000","last_peered":"2023-10-19T10:37:00.277993+0000","last_clean":"2023-10-19T10:37:00.277993+0000","last_became_active":"2023-10-19T10:36:59.350722+0000","last_became_peered":"2023-10-19T10:36:59.350722+0000","last_unstale":"2023-10-19T10:37:00.277993+0000","last_undegraded":"2023-10-19T10:37:00.277993+0000","last_fullsized":"2023-10-19T10:37:00.277993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097884+0000","last_change":"2023-10-19T10:36:55.108041+0000","last_active":"2023-10-19T10:37:00.097884+0000","last_peered":"2023-10-19T10:37:00.097884+0000","last_clean":"2023-10-19T10:37:00.097884+0000","last_became_active":"2023-10-19T10:36:55.107747+0000","last_became_peered":"2023-10-19T10:36:55.107747+0000","last_unstale":"2023-10-19T10:37:00.097884+0000","last_undegraded":"2023-10-19T10:37:00.097884+0000","last_fullsized":"2023-10-19T10:37:00.097884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.096738+0000","last_change":"2023-10-19T10:36:54.307533+0000","last_active":"2023-10-19T10:37:00.096738+0000","last_peered":"2023-10-19T10:37:00.096738+0000","last_clean":"2023-10-19T10:37:00.096738+0000","last_became_active":"2023-10-19T10:36:54.307237+0000","last_became_peered":"2023-10-19T10:36:54.307237+0000","last_unstale":"2023-10-19T10:37:00.096738+0000","last_undegraded":"2023-10-19T10:37:00.096738+0000","last_fullsized":"2023-10-19T10:37:00.096738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:57.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322887+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:37:00.322887+0000","last_peered":"2023-10-19T10:37:00.322887+0000","last_clean":"2023-10-19T10:37:00.322887+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:37:00.322887+0000","last_undegraded":"2023-10-19T10:37:00.322887+0000","last_fullsized":"2023-10-19T10:37:00.322887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:35:30.635889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324367+0000","last_change":"2023-10-19T10:36:59.349089+0000","last_active":"2023-10-19T10:37:00.324367+0000","last_peered":"2023-10-19T10:37:00.324367+0000","last_clean":"2023-10-19T10:37:00.324367+0000","last_became_active":"2023-10-19T10:36:59.348703+0000","last_became_peered":"2023-10-19T10:36:59.348703+0000","last_unstale":"2023-10-19T10:37:00.324367+0000","last_undegraded":"2023-10-19T10:37:00.324367+0000","last_fullsized":"2023-10-19T10:37:00.324367+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:03:04.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278271+0000","last_change":"2023-10-19T10:36:55.112250+0000","last_active":"2023-10-19T10:37:00.278271+0000","last_peered":"2023-10-19T10:37:00.278271+0000","last_clean":"2023-10-19T10:37:00.278271+0000","last_became_active":"2023-10-19T10:36:55.111996+0000","last_became_peered":"2023-10-19T10:36:55.111996+0000","last_unstale":"2023-10-19T10:37:00.278271+0000","last_undegraded":"2023-10-19T10:37:00.278271+0000","last_fullsized":"2023-10-19T10:37:00.278271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277550+0000","last_change":"2023-10-19T10:36:54.307098+0000","last_active":"2023-10-19T10:37:00.277550+0000","last_peered":"2023-10-19T10:37:00.277550+0000","last_clean":"2023-10-19T10:37:00.277550+0000","last_became_active":"2023-10-19T10:36:54.306769+0000","last_became_peered":"2023-10-19T10:36:54.306769+0000","last_unstale":"2023-10-19T10:37:00.277550+0000","last_undegraded":"2023-10-19T10:37:00.277550+0000","last_fullsized":"2023-10-19T10:37:00.277550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:51:13.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098430+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:37:00.098430+0000","last_peered":"2023-10-19T10:37:00.098430+0000","last_clean":"2023-10-19T10:37:00.098430+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:37:00.098430+0000","last_undegraded":"2023-10-19T10:37:00.098430+0000","last_fullsized":"2023-10-19T10:37:00.098430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324610+0000","last_change":"2023-10-19T10:36:59.357456+0000","last_active":"2023-10-19T10:37:00.324610+0000","last_peered":"2023-10-19T10:37:00.324610+0000","last_clean":"2023-10-19T10:37:00.324610+0000","last_became_active":"2023-10-19T10:36:59.357079+0000","last_became_peered":"2023-10-19T10:36:59.357079+0000","last_unstale":"2023-10-19T10:37:00.324610+0000","last_undegraded":"2023-10-19T10:37:00.324610+0000","last_fullsized":"2023-10-19T10:37:00.324610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:52.409495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323740+0000","last_change":"2023-10-19T10:36:55.113749+0000","last_active":"2023-10-19T10:37:00.323740+0000","last_peered":"2023-10-19T10:37:00.323740+0000","last_clean":"2023-10-19T10:37:00.323740+0000","last_became_active":"2023-10-19T10:36:55.113459+0000","last_became_peered":"2023-10-19T10:36:55.113459+0000","last_unstale":"2023-10-19T10:37:00.323740+0000","last_undegraded":"2023-10-19T10:37:00.323740+0000","last_fullsized":"2023-10-19T10:37:00.323740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:05:40.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323207+0000","last_change":"2023-10-19T10:36:54.309423+0000","last_active":"2023-10-19T10:37:00.323207+0000","last_peered":"2023-10-19T10:37:00.323207+0000","last_clean":"2023-10-19T10:37:00.323207+0000","last_became_active":"2023-10-19T10:36:54.309122+0000","last_became_peered":"2023-10-19T10:36:54.309122+0000","last_unstale":"2023-10-19T10:37:00.323207+0000","last_undegraded":"2023-10-19T10:37:00.323207+0000","last_fullsized":"2023-10-19T10:37:00.323207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:47.999447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324166+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:37:00.324166+0000","last_peered":"2023-10-19T10:37:00.324166+0000","last_clean":"2023-10-19T10:37:00.324166+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:37:00.324166+0000","last_undegraded":"2023-10-19T10:37:00.324166+0000","last_fullsized":"2023-10-19T10:37:00.324166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:43.964470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098587+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:37:00.098587+0000","last_peered":"2023-10-19T10:37:00.098587+0000","last_clean":"2023-10-19T10:37:00.098587+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:37:00.098587+0000","last_undegraded":"2023-10-19T10:37:00.098587+0000","last_fullsized":"2023-10-19T10:37:00.098587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:14.745066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098140+0000","last_change":"2023-10-19T10:36:59.359383+0000","last_active":"2023-10-19T10:37:00.098140+0000","last_peered":"2023-10-19T10:37:00.098140+0000","last_clean":"2023-10-19T10:37:00.098140+0000","last_became_active":"2023-10-19T10:36:59.358978+0000","last_became_peered":"2023-10-19T10:36:59.358978+0000","last_unstale":"2023-10-19T10:37:00.098140+0000","last_undegraded":"2023-10-19T10:37:00.098140+0000","last_fullsized":"2023-10-19T10:37:00.098140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:32:53.041602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097806+0000","last_change":"2023-10-19T10:36:55.111833+0000","last_active":"2023-10-19T10:37:00.097806+0000","last_peered":"2023-10-19T10:37:00.097806+0000","last_clean":"2023-10-19T10:37:00.097806+0000","last_became_active":"2023-10-19T10:36:55.111558+0000","last_became_peered":"2023-10-19T10:36:55.111558+0000","last_unstale":"2023-10-19T10:37:00.097806+0000","last_undegraded":"2023-10-19T10:37:00.097806+0000","last_fullsized":"2023-10-19T10:37:00.097806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:49.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323791+0000","last_change":"2023-10-19T10:36:54.304575+0000","last_active":"2023-10-19T10:37:00.323791+0000","last_peered":"2023-10-19T10:37:00.323791+0000","last_clean":"2023-10-19T10:37:00.323791+0000","last_became_active":"2023-10-19T10:36:54.304001+0000","last_became_peered":"2023-10-19T10:36:54.304001+0000","last_unstale":"2023-10-19T10:37:00.323791+0000","last_undegraded":"2023-10-19T10:37:00.323791+0000","last_fullsized":"2023-10-19T10:37:00.323791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099327+0000","last_change":"2023-10-19T10:36:59.347266+0000","last_active":"2023-10-19T10:37:00.099327+0000","last_peered":"2023-10-19T10:37:00.099327+0000","last_clean":"2023-10-19T10:37:00.099327+0000","last_became_active":"2023-10-19T10:36:59.346858+0000","last_became_peered":"2023-10-19T10:36:59.346858+0000","last_unstale":"2023-10-19T10:37:00.099327+0000","last_undegraded":"2023-10-19T10:37:00.099327+0000","last_fullsized":"2023-10-19T10:37:00.099327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:26:30.546568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099297+0000","last_change":"2023-10-19T10:36:55.107822+0000","last_active":"2023-10-19T10:37:00.099297+0000","last_peered":"2023-10-19T10:37:00.099297+0000","last_clean":"2023-10-19T10:37:00.099297+0000","last_became_active":"2023-10-19T10:36:55.107534+0000","last_became_peered":"2023-10-19T10:36:55.107534+0000","last_unstale":"2023-10-19T10:37:00.099297+0000","last_undegraded":"2023-10-19T10:37:00.099297+0000","last_fullsized":"2023-10-19T10:37:00.099297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:51:56.439816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098149+0000","last_change":"2023-10-19T10:36:54.306612+0000","last_active":"2023-10-19T10:37:00.098149+0000","last_peered":"2023-10-19T10:37:00.098149+0000","last_clean":"2023-10-19T10:37:00.098149+0000","last_became_active":"2023-10-19T10:36:54.306357+0000","last_became_peered":"2023-10-19T10:36:54.306357+0000","last_unstale":"2023-10-19T10:37:00.098149+0000","last_undegraded":"2023-10-19T10:37:00.098149+0000","last_fullsized":"2023-10-19T10:37:00.098149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:37:13.605874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324569+0000","last_change":"2023-10-19T10:36:59.352368+0000","last_active":"2023-10-19T10:37:00.324569+0000","last_peered":"2023-10-19T10:37:00.324569+0000","last_clean":"2023-10-19T10:37:00.324569+0000","last_became_active":"2023-10-19T10:36:59.352002+0000","last_became_peered":"2023-10-19T10:36:59.352002+0000","last_unstale":"2023-10-19T10:37:00.324569+0000","last_undegraded":"2023-10-19T10:37:00.324569+0000","last_fullsized":"2023-10-19T10:37:00.324569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:05.263646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099519+0000","last_change":"2023-10-19T10:36:55.110179+0000","last_active":"2023-10-19T10:37:00.099519+0000","last_peered":"2023-10-19T10:37:00.099519+0000","last_clean":"2023-10-19T10:37:00.099519+0000","last_became_active":"2023-10-19T10:36:55.109756+0000","last_became_peered":"2023-10-19T10:36:55.109756+0000","last_unstale":"2023-10-19T10:37:00.099519+0000","last_undegraded":"2023-10-19T10:37:00.099519+0000","last_fullsized":"2023-10-19T10:37:00.099519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:55:21.657235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323576+0000","last_change":"2023-10-19T10:36:54.306427+0000","last_active":"2023-10-19T10:37:00.323576+0000","last_peered":"2023-10-19T10:37:00.323576+0000","last_clean":"2023-10-19T10:37:00.323576+0000","last_became_active":"2023-10-19T10:36:54.306033+0000","last_became_peered":"2023-10-19T10:36:54.306033+0000","last_unstale":"2023-10-19T10:37:00.323576+0000","last_undegraded":"2023-10-19T10:37:00.323576+0000","last_fullsized":"2023-10-19T10:37:00.323576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:08:11.353560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097356+0000","last_change":"2023-10-19T10:36:59.353562+0000","last_active":"2023-10-19T10:37:00.097356+0000","last_peered":"2023-10-19T10:37:00.097356+0000","last_clean":"2023-10-19T10:37:00.097356+0000","last_became_active":"2023-10-19T10:36:59.353204+0000","last_became_peered":"2023-10-19T10:36:59.353204+0000","last_unstale":"2023-10-19T10:37:00.097356+0000","last_undegraded":"2023-10-19T10:37:00.097356+0000","last_fullsized":"2023-10-19T10:37:00.097356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099281+0000","last_change":"2023-10-19T10:36:55.115534+0000","last_active":"2023-10-19T10:37:00.099281+0000","last_peered":"2023-10-19T10:37:00.099281+0000","last_clean":"2023-10-19T10:37:00.099281+0000","last_became_active":"2023-10-19T10:36:55.114049+0000","last_became_peered":"2023-10-19T10:36:55.114049+0000","last_unstale":"2023-10-19T10:37:00.099281+0000","last_undegraded":"2023-10-19T10:37:00.099281+0000","last_fullsized":"2023-10-19T10:37:00.099281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:22:31.265043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323955+0000","last_change":"2023-10-19T10:36:54.310069+0000","last_active":"2023-10-19T10:37:00.323955+0000","last_peered":"2023-10-19T10:37:00.323955+0000","last_clean":"2023-10-19T10:37:00.323955+0000","last_became_active":"2023-10-19T10:36:54.309784+0000","last_became_peered":"2023-10-19T10:36:54.309784+0000","last_unstale":"2023-10-19T10:37:00.323955+0000","last_undegraded":"2023-10-19T10:37:00.323955+0000","last_fullsized":"2023-10-19T10:37:00.323955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323321+0000","last_change":"2023-10-19T10:36:59.351003+0000","last_active":"2023-10-19T10:37:00.323321+0000","last_peered":"2023-10-19T10:37:00.323321+0000","last_clean":"2023-10-19T10:37:00.323321+0000","last_became_active":"2023-10-19T10:36:59.349597+0000","last_became_peered":"2023-10-19T10:36:59.349597+0000","last_unstale":"2023-10-19T10:37:00.323321+0000","last_undegraded":"2023-10-19T10:37:00.323321+0000","last_fullsized":"2023-10-19T10:37:00.323321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:53:25.552100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277872+0000","last_change":"2023-10-19T10:36:55.118959+0000","last_active":"2023-10-19T10:37:00.277872+0000","last_peered":"2023-10-19T10:37:00.277872+0000","last_clean":"2023-10-19T10:37:00.277872+0000","last_became_active":"2023-10-19T10:36:55.116867+0000","last_became_peered":"2023-10-19T10:36:55.116867+0000","last_unstale":"2023-10-19T10:37:00.277872+0000","last_undegraded":"2023-10-19T10:37:00.277872+0000","last_fullsized":"2023-10-19T10:37:00.277872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324124+0000","last_change":"2023-10-19T10:36:54.305145+0000","last_active":"2023-10-19T10:37:00.324124+0000","last_peered":"2023-10-19T10:37:00.324124+0000","last_clean":"2023-10-19T10:37:00.324124+0000","last_became_active":"2023-10-19T10:36:54.304599+0000","last_became_peered":"2023-10-19T10:36:54.304599+0000","last_unstale":"2023-10-19T10:37:00.324124+0000","last_undegraded":"2023-10-19T10:37:00.324124+0000","last_fullsized":"2023-10-19T10:37:00.324124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:29.781334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323492+0000","last_change":"2023-10-19T10:36:59.348091+0000","last_active":"2023-10-19T10:37:00.323492+0000","last_peered":"2023-10-19T10:37:00.323492+0000","last_clean":"2023-10-19T10:37:00.323492+0000","last_became_active":"2023-10-19T10:36:59.346180+0000","last_became_peered":"2023-10-19T10:36:59.346180+0000","last_unstale":"2023-10-19T10:37:00.323492+0000","last_undegraded":"2023-10-19T10:37:00.323492+0000","last_fullsized":"2023-10-19T10:37:00.323492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:42:00.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323299+0000","last_change":"2023-10-19T10:36:55.114648+0000","last_active":"2023-10-19T10:37:00.323299+0000","last_peered":"2023-10-19T10:37:00.323299+0000","last_clean":"2023-10-19T10:37:00.323299+0000","last_became_active":"2023-10-19T10:36:55.114149+0000","last_became_peered":"2023-10-19T10:36:55.114149+0000","last_unstale":"2023-10-19T10:37:00.323299+0000","last_undegraded":"2023-10-19T10:37:00.323299+0000","last_fullsized":"2023-10-19T10:37:00.323299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:25:33.396338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098964+0000","last_change":"2023-10-19T10:36:54.309069+0000","last_active":"2023-10-19T10:37:00.098964+0000","last_peered":"2023-10-19T10:37:00.098964+0000","last_clean":"2023-10-19T10:37:00.098964+0000","last_became_active":"2023-10-19T10:36:54.308767+0000","last_became_peered":"2023-10-19T10:36:54.308767+0000","last_unstale":"2023-10-19T10:37:00.098964+0000","last_undegraded":"2023-10-19T10:37:00.098964+0000","last_fullsized":"2023-10-19T10:37:00.098964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:52:47.367341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323054+0000","last_change":"2023-10-19T10:36:59.349735+0000","last_active":"2023-10-19T10:37:00.323054+0000","last_peered":"2023-10-19T10:37:00.323054+0000","last_clean":"2023-10-19T10:37:00.323054+0000","last_became_active":"2023-10-19T10:36:59.349327+0000","last_became_peered":"2023-10-19T10:36:59.349327+0000","last_unstale":"2023-10-19T10:37:00.323054+0000","last_undegraded":"2023-10-19T10:37:00.323054+0000","last_fullsized":"2023-10-19T10:37:00.323054+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:51:57.082676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097759+0000","last_change":"2023-10-19T10:36:55.111295+0000","last_active":"2023-10-19T10:37:00.097759+0000","last_peered":"2023-10-19T10:37:00.097759+0000","last_clean":"2023-10-19T10:37:00.097759+0000","last_became_active":"2023-10-19T10:36:55.111016+0000","last_became_peered":"2023-10-19T10:36:55.111016+0000","last_unstale":"2023-10-19T10:37:00.097759+0000","last_undegraded":"2023-10-19T10:37:00.097759+0000","last_fullsized":"2023-10-19T10:37:00.097759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:01:05.121484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098592+0000","last_change":"2023-10-19T10:36:54.308800+0000","last_active":"2023-10-19T10:37:00.098592+0000","last_peered":"2023-10-19T10:37:00.098592+0000","last_clean":"2023-10-19T10:37:00.098592+0000","last_became_active":"2023-10-19T10:36:54.307919+0000","last_became_peered":"2023-10-19T10:36:54.307919+0000","last_unstale":"2023-10-19T10:37:00.098592+0000","last_undegraded":"2023-10-19T10:37:00.098592+0000","last_fullsized":"2023-10-19T10:37:00.098592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:00.862675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.322763+0000","last_change":"2023-10-19T10:36:59.350698+0000","last_active":"2023-10-19T10:37:00.322763+0000","last_peered":"2023-10-19T10:37:00.322763+0000","last_clean":"2023-10-19T10:37:00.322763+0000","last_became_active":"2023-10-19T10:36:59.350331+0000","last_became_peered":"2023-10-19T10:36:59.350331+0000","last_unstale":"2023-10-19T10:37:00.322763+0000","last_undegraded":"2023-10-19T10:37:00.322763+0000","last_fullsized":"2023-10-19T10:37:00.322763+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:30:44.911501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323455+0000","last_change":"2023-10-19T10:36:55.110357+0000","last_active":"2023-10-19T10:37:00.323455+0000","last_peered":"2023-10-19T10:37:00.323455+0000","last_clean":"2023-10-19T10:37:00.323455+0000","last_became_active":"2023-10-19T10:36:55.110058+0000","last_became_peered":"2023-10-19T10:36:55.110058+0000","last_unstale":"2023-10-19T10:37:00.323455+0000","last_undegraded":"2023-10-19T10:37:00.323455+0000","last_fullsized":"2023-10-19T10:37:00.323455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:39.760467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098977+0000","last_change":"2023-10-19T10:36:54.305854+0000","last_active":"2023-10-19T10:37:00.098977+0000","last_peered":"2023-10-19T10:37:00.098977+0000","last_clean":"2023-10-19T10:37:00.098977+0000","last_became_active":"2023-10-19T10:36:54.305559+0000","last_became_peered":"2023-10-19T10:36:54.305559+0000","last_unstale":"2023-10-19T10:37:00.098977+0000","last_undegraded":"2023-10-19T10:37:00.098977+0000","last_fullsized":"2023-10-19T10:37:00.098977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:50.004622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097838+0000","last_change":"2023-10-19T10:36:59.359644+0000","last_active":"2023-10-19T10:37:00.097838+0000","last_peered":"2023-10-19T10:37:00.097838+0000","last_clean":"2023-10-19T10:37:00.097838+0000","last_became_active":"2023-10-19T10:36:59.359083+0000","last_became_peered":"2023-10-19T10:36:59.359083+0000","last_unstale":"2023-10-19T10:37:00.097838+0000","last_undegraded":"2023-10-19T10:37:00.097838+0000","last_fullsized":"2023-10-19T10:37:00.097838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:43.798724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324477+0000","last_change":"2023-10-19T10:36:59.349191+0000","last_active":"2023-10-19T10:37:00.324477+0000","last_peered":"2023-10-19T10:37:00.324477+0000","last_clean":"2023-10-19T10:37:00.324477+0000","last_became_active":"2023-10-19T10:36:59.348388+0000","last_became_peered":"2023-10-19T10:36:59.348388+0000","last_unstale":"2023-10-19T10:37:00.324477+0000","last_undegraded":"2023-10-19T10:37:00.324477+0000","last_fullsized":"2023-10-19T10:37:00.324477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:50.929830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323413+0000","last_change":"2023-10-19T10:36:54.311108+0000","last_active":"2023-10-19T10:37:00.323413+0000","last_peered":"2023-10-19T10:37:00.323413+0000","last_clean":"2023-10-19T10:37:00.323413+0000","last_became_active":"2023-10-19T10:36:54.310804+0000","last_became_peered":"2023-10-19T10:36:54.310804+0000","last_unstale":"2023-10-19T10:37:00.323413+0000","last_undegraded":"2023-10-19T10:37:00.323413+0000","last_fullsized":"2023-10-19T10:37:00.323413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:42:24.153777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324424+0000","last_change":"2023-10-19T10:36:55.108985+0000","last_active":"2023-10-19T10:37:00.324424+0000","last_peered":"2023-10-19T10:37:00.324424+0000","last_clean":"2023-10-19T10:37:00.324424+0000","last_became_active":"2023-10-19T10:36:55.108726+0000","last_became_peered":"2023-10-19T10:36:55.108726+0000","last_unstale":"2023-10-19T10:37:00.324424+0000","last_undegraded":"2023-10-19T10:37:00.324424+0000","last_fullsized":"2023-10-19T10:37:00.324424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:54:29.622437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324136+0000","last_change":"2023-10-19T10:36:59.352844+0000","last_active":"2023-10-19T10:37:00.324136+0000","last_peered":"2023-10-19T10:37:00.324136+0000","last_clean":"2023-10-19T10:37:00.324136+0000","last_became_active":"2023-10-19T10:36:59.352451+0000","last_became_peered":"2023-10-19T10:36:59.352451+0000","last_unstale":"2023-10-19T10:37:00.324136+0000","last_undegraded":"2023-10-19T10:37:00.324136+0000","last_fullsized":"2023-10-19T10:37:00.324136+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:21.477523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098487+0000","last_change":"2023-10-19T10:36:54.310021+0000","last_active":"2023-10-19T10:37:00.098487+0000","last_peered":"2023-10-19T10:37:00.098487+0000","last_clean":"2023-10-19T10:37:00.098487+0000","last_became_active":"2023-10-19T10:36:54.309686+0000","last_became_peered":"2023-10-19T10:36:54.309686+0000","last_unstale":"2023-10-19T10:37:00.098487+0000","last_undegraded":"2023-10-19T10:37:00.098487+0000","last_fullsized":"2023-10-19T10:37:00.098487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:25.891199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278441+0000","last_change":"2023-10-19T10:36:55.118465+0000","last_active":"2023-10-19T10:37:00.278441+0000","last_peered":"2023-10-19T10:37:00.278441+0000","last_clean":"2023-10-19T10:37:00.278441+0000","last_became_active":"2023-10-19T10:36:55.116202+0000","last_became_peered":"2023-10-19T10:36:55.116202+0000","last_unstale":"2023-10-19T10:37:00.278441+0000","last_undegraded":"2023-10-19T10:37:00.278441+0000","last_fullsized":"2023-10-19T10:37:00.278441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277600+0000","last_change":"2023-10-19T10:36:54.306522+0000","last_active":"2023-10-19T10:37:00.277600+0000","last_peered":"2023-10-19T10:37:00.277600+0000","last_clean":"2023-10-19T10:37:00.277600+0000","last_became_active":"2023-10-19T10:36:54.306147+0000","last_became_peered":"2023-10-19T10:36:54.306147+0000","last_unstale":"2023-10-19T10:37:00.277600+0000","last_undegraded":"2023-10-19T10:37:00.277600+0000","last_fullsized":"2023-10-19T10:37:00.277600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:36:38.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324371+0000","last_change":"2023-10-19T10:36:55.110979+0000","last_active":"2023-10-19T10:37:00.324371+0000","last_peered":"2023-10-19T10:37:00.324371+0000","last_clean":"2023-10-19T10:37:00.324371+0000","last_became_active":"2023-10-19T10:36:55.109937+0000","last_became_peered":"2023-10-19T10:36:55.109937+0000","last_unstale":"2023-10-19T10:37:00.324371+0000","last_undegraded":"2023-10-19T10:37:00.324371+0000","last_fullsized":"2023-10-19T10:37:00.324371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323485+0000","last_change":"2023-10-19T10:36:59.355501+0000","last_active":"2023-10-19T10:37:00.323485+0000","last_peered":"2023-10-19T10:37:00.323485+0000","last_clean":"2023-10-19T10:37:00.323485+0000","last_became_active":"2023-10-19T10:36:59.353627+0000","last_became_peered":"2023-10-19T10:36:59.353627+0000","last_unstale":"2023-10-19T10:37:00.323485+0000","last_undegraded":"2023-10-19T10:37:00.323485+0000","last_fullsized":"2023-10-19T10:37:00.323485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:30.933353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098816+0000","last_change":"2023-10-19T10:36:54.308041+0000","last_active":"2023-10-19T10:37:00.098816+0000","last_peered":"2023-10-19T10:37:00.098816+0000","last_clean":"2023-10-19T10:37:00.098816+0000","last_became_active":"2023-10-19T10:36:54.307752+0000","last_became_peered":"2023-10-19T10:36:54.307752+0000","last_unstale":"2023-10-19T10:37:00.098816+0000","last_undegraded":"2023-10-19T10:37:00.098816+0000","last_fullsized":"2023-10-19T10:37:00.098816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:53.280708+0000","last_clean_scrub_stamp":"2023-10-19T10:36:53.280708+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:08.374742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323551+0000","last_change":"2023-10-19T10:36:55.115058+0000","last_active":"2023-10-19T10:37:00.323551+0000","last_peered":"2023-10-19T10:37:00.323551+0000","last_clean":"2023-10-19T10:37:00.323551+0000","last_became_active":"2023-10-19T10:36:55.114729+0000","last_became_peered":"2023-10-19T10:36:55.114729+0000","last_unstale":"2023-10-19T10:37:00.323551+0000","last_undegraded":"2023-10-19T10:37:00.323551+0000","last_fullsized":"2023-10-19T10:37:00.323551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324110+0000","last_change":"2023-10-19T10:36:59.356762+0000","last_active":"2023-10-19T10:37:00.324110+0000","last_peered":"2023-10-19T10:37:00.324110+0000","last_clean":"2023-10-19T10:37:00.324110+0000","last_became_active":"2023-10-19T10:36:59.356422+0000","last_became_peered":"2023-10-19T10:36:59.356422+0000","last_unstale":"2023-10-19T10:37:00.324110+0000","last_undegraded":"2023-10-19T10:37:00.324110+0000","last_fullsized":"2023-10-19T10:37:00.324110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:58.322862+0000","last_clean_scrub_stamp":"2023-10-19T10:36:58.322862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:30.887325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323028+0000","last_change":"2023-10-19T10:36:55.108618+0000","last_active":"2023-10-19T10:37:00.323028+0000","last_peered":"2023-10-19T10:37:00.323028+0000","last_clean":"2023-10-19T10:37:00.323028+0000","last_became_active":"2023-10-19T10:36:55.108364+0000","last_became_peered":"2023-10-19T10:36:55.108364+0000","last_unstale":"2023-10-19T10:37:00.323028+0000","last_undegraded":"2023-10-19T10:37:00.323028+0000","last_fullsized":"2023-10-19T10:37:00.323028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323784+0000","last_change":"2023-10-19T10:36:55.114936+0000","last_active":"2023-10-19T10:37:00.323784+0000","last_peered":"2023-10-19T10:37:00.323784+0000","last_clean":"2023-10-19T10:37:00.323784+0000","last_became_active":"2023-10-19T10:36:55.114673+0000","last_became_peered":"2023-10-19T10:36:55.114673+0000","last_unstale":"2023-10-19T10:37:00.323784+0000","last_undegraded":"2023-10-19T10:37:00.323784+0000","last_fullsized":"2023-10-19T10:37:00.323784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:27:01.225389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098152+0000","last_change":"2023-10-19T10:36:55.107446+0000","last_active":"2023-10-19T10:37:00.098152+0000","last_peered":"2023-10-19T10:37:00.098152+0000","last_clean":"2023-10-19T10:37:00.098152+0000","last_became_active":"2023-10-19T10:36:55.107185+0000","last_became_peered":"2023-10-19T10:36:55.107185+0000","last_unstale":"2023-10-19T10:37:00.098152+0000","last_undegraded":"2023-10-19T10:37:00.098152+0000","last_fullsized":"2023-10-19T10:37:00.098152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:36:13.224759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323701+0000","last_change":"2023-10-19T10:36:55.108586+0000","last_active":"2023-10-19T10:37:00.323701+0000","last_peered":"2023-10-19T10:37:00.323701+0000","last_clean":"2023-10-19T10:37:00.323701+0000","last_became_active":"2023-10-19T10:36:55.108308+0000","last_became_peered":"2023-10-19T10:36:55.108308+0000","last_unstale":"2023-10-19T10:37:00.323701+0000","last_undegraded":"2023-10-19T10:37:00.323701+0000","last_fullsized":"2023-10-19T10:37:00.323701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:20:09.378639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097485+0000","last_change":"2023-10-19T10:36:55.112395+0000","last_active":"2023-10-19T10:37:00.097485+0000","last_peered":"2023-10-19T10:37:00.097485+0000","last_clean":"2023-10-19T10:37:00.097485+0000","last_became_active":"2023-10-19T10:36:55.112101+0000","last_became_peered":"2023-10-19T10:36:55.112101+0000","last_unstale":"2023-10-19T10:37:00.097485+0000","last_undegraded":"2023-10-19T10:37:00.097485+0000","last_fullsized":"2023-10-19T10:37:00.097485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:23:26.849068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324215+0000","last_change":"2023-10-19T10:36:55.114054+0000","last_active":"2023-10-19T10:37:00.324215+0000","last_peered":"2023-10-19T10:37:00.324215+0000","last_clean":"2023-10-19T10:37:00.324215+0000","last_became_active":"2023-10-19T10:36:55.113749+0000","last_became_peered":"2023-10-19T10:36:55.113749+0000","last_unstale":"2023-10-19T10:37:00.324215+0000","last_undegraded":"2023-10-19T10:37:00.324215+0000","last_fullsized":"2023-10-19T10:37:00.324215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097817+0000","last_change":"2023-10-19T10:36:55.109364+0000","last_active":"2023-10-19T10:37:00.097817+0000","last_peered":"2023-10-19T10:37:00.097817+0000","last_clean":"2023-10-19T10:37:00.097817+0000","last_became_active":"2023-10-19T10:36:55.109110+0000","last_became_peered":"2023-10-19T10:36:55.109110+0000","last_unstale":"2023-10-19T10:37:00.097817+0000","last_undegraded":"2023-10-19T10:37:00.097817+0000","last_fullsized":"2023-10-19T10:37:00.097817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:26.895096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097790+0000","last_change":"2023-10-19T10:36:55.111296+0000","last_active":"2023-10-19T10:37:00.097790+0000","last_peered":"2023-10-19T10:37:00.097790+0000","last_clean":"2023-10-19T10:37:00.097790+0000","last_became_active":"2023-10-19T10:36:55.111042+0000","last_became_peered":"2023-10-19T10:36:55.111042+0000","last_unstale":"2023-10-19T10:37:00.097790+0000","last_undegraded":"2023-10-19T10:37:00.097790+0000","last_fullsized":"2023-10-19T10:37:00.097790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098307+0000","last_change":"2023-10-19T10:36:55.117568+0000","last_active":"2023-10-19T10:37:00.098307+0000","last_peered":"2023-10-19T10:37:00.098307+0000","last_clean":"2023-10-19T10:37:00.098307+0000","last_became_active":"2023-10-19T10:36:55.117281+0000","last_became_peered":"2023-10-19T10:36:55.117281+0000","last_unstale":"2023-10-19T10:37:00.098307+0000","last_undegraded":"2023-10-19T10:37:00.098307+0000","last_fullsized":"2023-10-19T10:37:00.098307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:38:56.721204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323081+0000","last_change":"2023-10-19T10:36:55.114123+0000","last_active":"2023-10-19T10:37:00.323081+0000","last_peered":"2023-10-19T10:37:00.323081+0000","last_clean":"2023-10-19T10:37:00.323081+0000","last_became_active":"2023-10-19T10:36:55.113832+0000","last_became_peered":"2023-10-19T10:36:55.113832+0000","last_unstale":"2023-10-19T10:37:00.323081+0000","last_undegraded":"2023-10-19T10:37:00.323081+0000","last_fullsized":"2023-10-19T10:37:00.323081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.277576+0000","last_change":"2023-10-19T10:36:55.115474+0000","last_active":"2023-10-19T10:37:00.277576+0000","last_peered":"2023-10-19T10:37:00.277576+0000","last_clean":"2023-10-19T10:37:00.277576+0000","last_became_active":"2023-10-19T10:36:55.115133+0000","last_became_peered":"2023-10-19T10:36:55.115133+0000","last_unstale":"2023-10-19T10:37:00.277576+0000","last_undegraded":"2023-10-19T10:37:00.277576+0000","last_fullsized":"2023-10-19T10:37:00.277576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098770+0000","last_change":"2023-10-19T10:36:55.117512+0000","last_active":"2023-10-19T10:37:00.098770+0000","last_peered":"2023-10-19T10:37:00.098770+0000","last_clean":"2023-10-19T10:37:00.098770+0000","last_became_active":"2023-10-19T10:36:55.117143+0000","last_became_peered":"2023-10-19T10:36:55.117143+0000","last_unstale":"2023-10-19T10:37:00.098770+0000","last_undegraded":"2023-10-19T10:37:00.098770+0000","last_fullsized":"2023-10-19T10:37:00.098770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:46.891573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099100+0000","last_change":"2023-10-19T10:36:55.106989+0000","last_active":"2023-10-19T10:37:00.099100+0000","last_peered":"2023-10-19T10:37:00.099100+0000","last_clean":"2023-10-19T10:37:00.099100+0000","last_became_active":"2023-10-19T10:36:55.106735+0000","last_became_peered":"2023-10-19T10:36:55.106735+0000","last_unstale":"2023-10-19T10:37:00.099100+0000","last_undegraded":"2023-10-19T10:37:00.099100+0000","last_fullsized":"2023-10-19T10:37:00.099100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:49:54.003217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324208+0000","last_change":"2023-10-19T10:36:55.107156+0000","last_active":"2023-10-19T10:37:00.324208+0000","last_peered":"2023-10-19T10:37:00.324208+0000","last_clean":"2023-10-19T10:37:00.324208+0000","last_became_active":"2023-10-19T10:36:55.106636+0000","last_became_peered":"2023-10-19T10:36:55.106636+0000","last_unstale":"2023-10-19T10:37:00.324208+0000","last_undegraded":"2023-10-19T10:37:00.324208+0000","last_fullsized":"2023-10-19T10:37:00.324208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:02:38.159925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098995+0000","last_change":"2023-10-19T10:36:55.102460+0000","last_active":"2023-10-19T10:37:00.098995+0000","last_peered":"2023-10-19T10:37:00.098995+0000","last_clean":"2023-10-19T10:37:00.098995+0000","last_became_active":"2023-10-19T10:36:55.102197+0000","last_became_peered":"2023-10-19T10:36:55.102197+0000","last_unstale":"2023-10-19T10:37:00.098995+0000","last_undegraded":"2023-10-19T10:37:00.098995+0000","last_fullsized":"2023-10-19T10:37:00.098995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:18:06.083802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278315+0000","last_change":"2023-10-19T10:36:55.118142+0000","last_active":"2023-10-19T10:37:00.278315+0000","last_peered":"2023-10-19T10:37:00.278315+0000","last_clean":"2023-10-19T10:37:00.278315+0000","last_became_active":"2023-10-19T10:36:55.116529+0000","last_became_peered":"2023-10-19T10:36:55.116529+0000","last_unstale":"2023-10-19T10:37:00.278315+0000","last_undegraded":"2023-10-19T10:37:00.278315+0000","last_fullsized":"2023-10-19T10:37:00.278315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:49:26.508553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097588+0000","last_change":"2023-10-19T10:36:55.110137+0000","last_active":"2023-10-19T10:37:00.097588+0000","last_peered":"2023-10-19T10:37:00.097588+0000","last_clean":"2023-10-19T10:37:00.097588+0000","last_became_active":"2023-10-19T10:36:55.109003+0000","last_became_peered":"2023-10-19T10:36:55.109003+0000","last_unstale":"2023-10-19T10:37:00.097588+0000","last_undegraded":"2023-10-19T10:37:00.097588+0000","last_fullsized":"2023-10-19T10:37:00.097588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:16:38.218180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.098351+0000","last_change":"2023-10-19T10:36:55.115362+0000","last_active":"2023-10-19T10:37:00.098351+0000","last_peered":"2023-10-19T10:37:00.098351+0000","last_clean":"2023-10-19T10:37:00.098351+0000","last_became_active":"2023-10-19T10:36:55.114806+0000","last_became_peered":"2023-10-19T10:36:55.114806+0000","last_unstale":"2023-10-19T10:37:00.098351+0000","last_undegraded":"2023-10-19T10:37:00.098351+0000","last_fullsized":"2023-10-19T10:37:00.098351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323850+0000","last_change":"2023-10-19T10:36:55.106573+0000","last_active":"2023-10-19T10:37:00.323850+0000","last_peered":"2023-10-19T10:37:00.323850+0000","last_clean":"2023-10-19T10:37:00.323850+0000","last_became_active":"2023-10-19T10:36:55.106293+0000","last_became_peered":"2023-10-19T10:36:55.106293+0000","last_unstale":"2023-10-19T10:37:00.323850+0000","last_undegraded":"2023-10-19T10:37:00.323850+0000","last_fullsized":"2023-10-19T10:37:00.323850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:02:03.357075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323255+0000","last_change":"2023-10-19T10:36:55.116400+0000","last_active":"2023-10-19T10:37:00.323255+0000","last_peered":"2023-10-19T10:37:00.323255+0000","last_clean":"2023-10-19T10:37:00.323255+0000","last_became_active":"2023-10-19T10:36:55.116142+0000","last_became_peered":"2023-10-19T10:36:55.116142+0000","last_unstale":"2023-10-19T10:37:00.323255+0000","last_undegraded":"2023-10-19T10:37:00.323255+0000","last_fullsized":"2023-10-19T10:37:00.323255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:00:11.139684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323751+0000","last_change":"2023-10-19T10:36:55.102152+0000","last_active":"2023-10-19T10:37:00.323751+0000","last_peered":"2023-10-19T10:37:00.323751+0000","last_clean":"2023-10-19T10:37:00.323751+0000","last_became_active":"2023-10-19T10:36:55.101878+0000","last_became_peered":"2023-10-19T10:36:55.101878+0000","last_unstale":"2023-10-19T10:37:00.323751+0000","last_undegraded":"2023-10-19T10:37:00.323751+0000","last_fullsized":"2023-10-19T10:37:00.323751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:27:50.253900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099451+0000","last_change":"2023-10-19T10:36:55.111165+0000","last_active":"2023-10-19T10:37:00.099451+0000","last_peered":"2023-10-19T10:37:00.099451+0000","last_clean":"2023-10-19T10:37:00.099451+0000","last_became_active":"2023-10-19T10:36:55.110828+0000","last_became_peered":"2023-10-19T10:36:55.110828+0000","last_unstale":"2023-10-19T10:37:00.099451+0000","last_undegraded":"2023-10-19T10:37:00.099451+0000","last_fullsized":"2023-10-19T10:37:00.099451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:19:39.989618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324638+0000","last_change":"2023-10-19T10:36:55.110928+0000","last_active":"2023-10-19T10:37:00.324638+0000","last_peered":"2023-10-19T10:37:00.324638+0000","last_clean":"2023-10-19T10:37:00.324638+0000","last_became_active":"2023-10-19T10:36:55.110647+0000","last_became_peered":"2023-10-19T10:36:55.110647+0000","last_unstale":"2023-10-19T10:37:00.324638+0000","last_undegraded":"2023-10-19T10:37:00.324638+0000","last_fullsized":"2023-10-19T10:37:00.324638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:45:17.439266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278188+0000","last_change":"2023-10-19T10:36:55.118334+0000","last_active":"2023-10-19T10:37:00.278188+0000","last_peered":"2023-10-19T10:37:00.278188+0000","last_clean":"2023-10-19T10:37:00.278188+0000","last_became_active":"2023-10-19T10:36:55.116957+0000","last_became_peered":"2023-10-19T10:36:55.116957+0000","last_unstale":"2023-10-19T10:37:00.278188+0000","last_undegraded":"2023-10-19T10:37:00.278188+0000","last_fullsized":"2023-10-19T10:37:00.278188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:21:30.785232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323040+0000","last_change":"2023-10-19T10:36:55.110747+0000","last_active":"2023-10-19T10:37:00.323040+0000","last_peered":"2023-10-19T10:37:00.323040+0000","last_clean":"2023-10-19T10:37:00.323040+0000","last_became_active":"2023-10-19T10:36:55.110453+0000","last_became_peered":"2023-10-19T10:36:55.110453+0000","last_unstale":"2023-10-19T10:37:00.323040+0000","last_undegraded":"2023-10-19T10:37:00.323040+0000","last_fullsized":"2023-10-19T10:37:00.323040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:17:45.758636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097980+0000","last_change":"2023-10-19T10:36:55.111146+0000","last_active":"2023-10-19T10:37:00.097980+0000","last_peered":"2023-10-19T10:37:00.097980+0000","last_clean":"2023-10-19T10:37:00.097980+0000","last_became_active":"2023-10-19T10:36:55.110894+0000","last_became_peered":"2023-10-19T10:36:55.110894+0000","last_unstale":"2023-10-19T10:37:00.097980+0000","last_undegraded":"2023-10-19T10:37:00.097980+0000","last_fullsized":"2023-10-19T10:37:00.097980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:08:42.204020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.097815+0000","last_change":"2023-10-19T10:36:55.109335+0000","last_active":"2023-10-19T10:37:00.097815+0000","last_peered":"2023-10-19T10:37:00.097815+0000","last_clean":"2023-10-19T10:37:00.097815+0000","last_became_active":"2023-10-19T10:36:55.109002+0000","last_became_peered":"2023-10-19T10:36:55.109002+0000","last_unstale":"2023-10-19T10:37:00.097815+0000","last_undegraded":"2023-10-19T10:37:00.097815+0000","last_fullsized":"2023-10-19T10:37:00.097815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:32:10.062310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.278344+0000","last_change":"2023-10-19T10:36:55.119014+0000","last_active":"2023-10-19T10:37:00.278344+0000","last_peered":"2023-10-19T10:37:00.278344+0000","last_clean":"2023-10-19T10:37:00.278344+0000","last_became_active":"2023-10-19T10:36:55.118470+0000","last_became_peered":"2023-10-19T10:36:55.118470+0000","last_unstale":"2023-10-19T10:37:00.278344+0000","last_undegraded":"2023-10-19T10:37:00.278344+0000","last_fullsized":"2023-10-19T10:37:00.278344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:32:01.866572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.323295+0000","last_change":"2023-10-19T10:36:55.114470+0000","last_active":"2023-10-19T10:37:00.323295+0000","last_peered":"2023-10-19T10:37:00.323295+0000","last_clean":"2023-10-19T10:37:00.323295+0000","last_became_active":"2023-10-19T10:36:55.113809+0000","last_became_peered":"2023-10-19T10:36:55.113809+0000","last_unstale":"2023-10-19T10:37:00.323295+0000","last_undegraded":"2023-10-19T10:37:00.323295+0000","last_fullsized":"2023-10-19T10:37:00.323295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:22:52.211277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.099367+0000","last_change":"2023-10-19T10:36:55.113901+0000","last_active":"2023-10-19T10:37:00.099367+0000","last_peered":"2023-10-19T10:37:00.099367+0000","last_clean":"2023-10-19T10:37:00.099367+0000","last_became_active":"2023-10-19T10:36:55.113629+0000","last_became_peered":"2023-10-19T10:36:55.113629+0000","last_unstale":"2023-10-19T10:37:00.099367+0000","last_undegraded":"2023-10-19T10:37:00.099367+0000","last_fullsized":"2023-10-19T10:37:00.099367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:20:21.320100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324574+0000","last_change":"2023-10-19T10:36:55.112724+0000","last_active":"2023-10-19T10:37:00.324574+0000","last_peered":"2023-10-19T10:37:00.324574+0000","last_clean":"2023-10-19T10:37:00.324574+0000","last_became_active":"2023-10-19T10:36:55.112307+0000","last_became_peered":"2023-10-19T10:36:55.112307+0000","last_unstale":"2023-10-19T10:37:00.324574+0000","last_undegraded":"2023-10-19T10:37:00.324574+0000","last_fullsized":"2023-10-19T10:37:00.324574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:05:51.008519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-19T10:37:00.324311+0000","last_change":"2023-10-19T10:36:55.107659+0000","last_active":"2023-10-19T10:37:00.324311+0000","last_peered":"2023-10-19T10:37:00.324311+0000","last_clean":"2023-10-19T10:37:00.324311+0000","last_became_active":"2023-10-19T10:36:55.107382+0000","last_became_peered":"2023-10-19T10:36:55.107382+0000","last_unstale":"2023-10-19T10:37:00.324311+0000","last_undegraded":"2023-10-19T10:37:00.324311+0000","last_fullsized":"2023-10-19T10:37:00.324311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:54.284605+0000","last_clean_scrub_stamp":"2023-10-19T10:36:54.284605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:38:44.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":294912,"data_stored":119711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":290816,"data_stored":119621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":888832,"data_stored":710571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:37:25.166 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-19T10:37:25.166 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-19T10:37:25.166 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-19T10:37:25.167 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-19T10:37:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.302+0000 7faf8749d640 1 Processor -- start 2023-10-19T10:37:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 -- start start 2023-10-19T10:37:25.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faf8015c620 0x7faf800b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faf800b82f0 0x7faf800cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b55c0 0x7faf800cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800ccc10 con 0x7faf8015c620 2023-10-19T10:37:25.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800cda30 con 0x7faf800b82f0 2023-10-19T10:37:25.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.303+0000 7faf8749d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf80161110 con 0x7faf800b55c0 2023-10-19T10:37:25.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faf8015c620 0x7faf800b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faf8015c620 0x7faf800b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39278/0 (socket says 172.21.15.50:39278) 2023-10-19T10:37:25.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 -- 172.21.15.50:0/4087364947 learned_addr learned my addr 172.21.15.50:0/4087364947 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:25.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf84a38640 1 --2- 172.21.15.50:0/4087364947 >> v2:172.21.15.183:3300/0 conn(0x7faf800b82f0 0x7faf800cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85a3a640 1 --2- 172.21.15.50:0/4087364947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b55c0 0x7faf800cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 -- 172.21.15.50:0/4087364947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b55c0 msgr2=0x7faf800cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:25.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 --2- 172.21.15.50:0/4087364947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b55c0 0x7faf800cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 -- 172.21.15.50:0/4087364947 >> v2:172.21.15.183:3300/0 conn(0x7faf800b82f0 msgr2=0x7faf800cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 --2- 172.21.15.50:0/4087364947 >> v2:172.21.15.183:3300/0 conn(0x7faf800b82f0 0x7faf800cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.304+0000 7faf85239640 1 -- 172.21.15.50:0/4087364947 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf800cce20 con 0x7faf8015c620 2023-10-19T10:37:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf85239640 1 --2- 172.21.15.50:0/4087364947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf8015c620 0x7faf800b7db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faf800b50e0 tx=0x7faf7802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439a11db61ad863e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf7ffff640 1 -- 172.21.15.50:0/4087364947 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78031870 con 0x7faf8015c620 2023-10-19T10:37:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf7ffff640 1 -- 172.21.15.50:0/4087364947 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf780319d0 con 0x7faf8015c620 2023-10-19T10:37:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf7ffff640 1 -- 172.21.15.50:0/4087364947 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78031d60 con 0x7faf8015c620 2023-10-19T10:37:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 -- 172.21.15.50:0/4087364947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf8015c620 msgr2=0x7faf800b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 --2- 172.21.15.50:0/4087364947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf8015c620 0x7faf800b7db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faf800b50e0 tx=0x7faf7802b910 comp rx=0 tx=0).stop 2023-10-19T10:37:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 -- 172.21.15.50:0/4087364947 shutdown_connections 2023-10-19T10:37:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 --2- 172.21.15.50:0/4087364947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b55c0 0x7faf800cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 --2- 172.21.15.50:0/4087364947 >> v2:172.21.15.183:3300/0 conn(0x7faf800b82f0 0x7faf800cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 --2- 172.21.15.50:0/4087364947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf8015c620 0x7faf800b7db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 -- 172.21.15.50:0/4087364947 >> 172.21.15.50:0/4087364947 conn(0x7faf80071110 msgr2=0x7faf800714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.305+0000 7faf8749d640 1 -- 172.21.15.50:0/4087364947 shutdown_connections 2023-10-19T10:37:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.306+0000 7faf8749d640 1 -- 172.21.15.50:0/4087364947 wait complete. 2023-10-19T10:37:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 Processor -- start 2023-10-19T10:37:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 -- start start 2023-10-19T10:37:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 0x7faf800cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b82f0 0x7faf800c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf8015c620 0x7faf801f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800ca2d0 con 0x7faf800b55c0 2023-10-19T10:37:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800ca5b0 con 0x7faf8015c620 2023-10-19T10:37:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf8749d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf800ca890 con 0x7faf800b82f0 2023-10-19T10:37:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf85239640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 0x7faf800cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf85239640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 0x7faf800cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39280/0 (socket says 172.21.15.50:39280) 2023-10-19T10:37:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.307+0000 7faf85239640 1 -- 172.21.15.50:0/848173086 learned_addr learned my addr 172.21.15.50:0/848173086 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85a3a640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf8015c620 0x7faf801f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf84a38640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b82f0 0x7faf800c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85239640 1 -- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b82f0 msgr2=0x7faf800c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85239640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b82f0 0x7faf800c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85239640 1 -- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf8015c620 msgr2=0x7faf801f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85239640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf8015c620 0x7faf801f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85239640 1 -- 172.21.15.50:0/848173086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780094d0 con 0x7faf800b55c0 2023-10-19T10:37:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85a3a640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf8015c620 0x7faf801f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf84a38640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b82f0 0x7faf800c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:25.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf85239640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 0x7faf800cb370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf7802b950 tx=0x7faf780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:25.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7803ac60 con 0x7faf800b55c0 2023-10-19T10:37:25.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7803adc0 con 0x7faf800b55c0 2023-10-19T10:37:25.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf800cab70 con 0x7faf800b55c0 2023-10-19T10:37:25.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78044630 con 0x7faf800b55c0 2023-10-19T10:37:25.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.308+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf800c9fc0 con 0x7faf800b55c0 2023-10-19T10:37:25.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.309+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf800cb090 con 0x7faf800b55c0 2023-10-19T10:37:25.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.310+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf7804d450 con 0x7faf800b55c0 2023-10-19T10:37:25.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.310+0000 7faf7dffb640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf6c06a050 0x7faf6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.310+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faf7802d040 con 0x7faf800b55c0 2023-10-19T10:37:25.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.310+0000 7faf84a38640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf6c06a050 0x7faf6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.311+0000 7faf84a38640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf6c06a050 0x7faf6c06c510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faf800ccb40 tx=0x7faf74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:25.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.315+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7faf800cb090 con 0x7faf800b55c0 2023-10-19T10:37:25.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.527+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faf800cb090 con 0x7faf800b55c0 2023-10-19T10:37:25.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.528+0000 7faf7dffb640 1 -- 172.21.15.50:0/848173086 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7faf78035040 con 0x7faf800b55c0 2023-10-19T10:37:25.531 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:25.531 INFO:teuthology.orchestra.run.smithi050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-19T10:37:25.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf6c06a050 msgr2=0x7faf6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf6c06a050 0x7faf6c06c510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faf800ccb40 tx=0x7faf74008040 comp rx=0 tx=0).stop 2023-10-19T10:37:25.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 msgr2=0x7faf800cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 0x7faf800cb370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf7802b950 tx=0x7faf780046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 shutdown_connections 2023-10-19T10:37:25.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faf6c06a050 0x7faf6c06c510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf8015c620 0x7faf801f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf800b82f0 0x7faf800c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 --2- 172.21.15.50:0/848173086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf800b55c0 0x7faf800cb370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 >> 172.21.15.50:0/848173086 conn(0x7faf80071110 msgr2=0x7faf800b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:25.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 shutdown_connections 2023-10-19T10:37:25.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.530+0000 7faf8749d640 1 -- 172.21.15.50:0/848173086 wait complete. 2023-10-19T10:37:25.543 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-19T10:37:25.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:25.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.647+0000 7f87939a0640 1 Processor -- start 2023-10-19T10:37:25.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.647+0000 7f87939a0640 1 -- start start 2023-10-19T10:37:25.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.648+0000 7f87939a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.648+0000 7f87939a0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c06aa30 con 0x7f878c15c950 2023-10-19T10:37:25.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.648+0000 7f87939a0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c06b2d0 con 0x7f878c0cd920 2023-10-19T10:37:25.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.648+0000 7f87939a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0ce090 con 0x7f878c15c570 2023-10-19T10:37:25.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.648+0000 7f8790f3b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f878c15c950 0x7f878c06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33642/0 (socket says 172.21.15.50:33642) 2023-10-19T10:37:25.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.648+0000 7f8790f3b640 1 -- 172.21.15.50:0/4090839533 learned_addr learned my addr 172.21.15.50:0/4090839533 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:25.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f879173c640 1 --2- 172.21.15.50:0/4090839533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544573422 0 0) 0x7f878c06aa30 con 0x7f878c15c950 2023-10-19T10:37:25.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8774003520 con 0x7f878c15c950 2023-10-19T10:37:25.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181876347 0 0) 0x7f878c06b2d0 con 0x7f878c0cd920 2023-10-19T10:37:25.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f878c06aa30 con 0x7f878c0cd920 2023-10-19T10:37:25.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f879173c640 1 -- 172.21.15.50:0/4090839533 >> v1:172.21.15.183:6789/0 conn(0x7f878c0cd920 legacy=0x7f878c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f879173c640 1 -- 172.21.15.50:0/4090839533 >> v1:172.21.15.50:6789/0 conn(0x7f878c15c950 legacy=0x7f878c06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.649+0000 7f879173c640 1 -- 172.21.15.50:0/4090839533 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c06ac80 con 0x7f878c15c570 2023-10-19T10:37:25.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f879173c640 1 --2- 172.21.15.50:0/4090839533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c0d0e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f87840097c0 tx=0x7f878402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37a2b82e5976228 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:25.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784031800 con 0x7f878c15c570 2023-10-19T10:37:25.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784031960 con 0x7f878c15c570 2023-10-19T10:37:25.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f878bfff640 1 -- 172.21.15.50:0/4090839533 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784031cd0 con 0x7f878c15c570 2023-10-19T10:37:25.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f87939a0640 1 -- 172.21.15.50:0/4090839533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 msgr2=0x7f878c0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f87939a0640 1 --2- 172.21.15.50:0/4090839533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c0d0e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f87840097c0 tx=0x7f878402b8a0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f87939a0640 1 -- 172.21.15.50:0/4090839533 shutdown_connections 2023-10-19T10:37:25.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f87939a0640 1 --2- 172.21.15.50:0/4090839533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c0d0e30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.650+0000 7f87939a0640 1 -- 172.21.15.50:0/4090839533 >> 172.21.15.50:0/4090839533 conn(0x7f878c089ff0 msgr2=0x7f878c09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:25.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.651+0000 7f87939a0640 1 -- 172.21.15.50:0/4090839533 shutdown_connections 2023-10-19T10:37:25.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.651+0000 7f87939a0640 1 -- 172.21.15.50:0/4090839533 wait complete. 2023-10-19T10:37:25.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 Processor -- start 2023-10-19T10:37:25.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 -- start start 2023-10-19T10:37:25.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f878c0cd920 0x7f878c1f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 0x7f878c15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c15d980 con 0x7f878c15c950 2023-10-19T10:37:25.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c15dc60 con 0x7f878c0cd920 2023-10-19T10:37:25.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f87939a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c15df40 con 0x7f878c15c570 2023-10-19T10:37:25.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.652+0000 7f8791f3d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 0x7f878c15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 0x7f878c15fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39292/0 (socket says 172.21.15.50:39292) 2023-10-19T10:37:25.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 -- 172.21.15.50:0/2391928053 learned_addr learned my addr 172.21.15.50:0/2391928053 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:25.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f879173c640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f878c0cd920 0x7f878c1f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8790f3b640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 -- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 msgr2=0x7f878c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 -- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f878c0cd920 msgr2=0x7f878c1f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f878c0cd920 0x7f878c1f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 -- 172.21.15.50:0/2391928053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840093a0 con 0x7f878c15c950 2023-10-19T10:37:25.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8790f3b640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:25.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8791f3d640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 0x7f878c15fa20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c700 tx=0x7f877c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:25.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c003b70 con 0x7f878c15c950 2023-10-19T10:37:25.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c15e220 con 0x7f878c15c950 2023-10-19T10:37:25.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c003cd0 con 0x7f878c15c950 2023-10-19T10:37:25.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c0052d0 con 0x7f878c15c950 2023-10-19T10:37:25.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.653+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c15e7e0 con 0x7f878c15c950 2023-10-19T10:37:25.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.654+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c1f1c70 con 0x7f878c15c950 2023-10-19T10:37:25.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.655+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f877c005430 con 0x7f878c15c950 2023-10-19T10:37:25.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.655+0000 7f8789ffb640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f877806a050 0x7f877806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:25.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.655+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f877c088530 con 0x7f878c15c950 2023-10-19T10:37:25.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.655+0000 7f879173c640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f877806a050 0x7f877806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:25.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.656+0000 7f879173c640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f877806a050 0x7f877806c510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8784036000 tx=0x7f8784008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:25.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.660+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f877c053940 con 0x7f878c15c950 2023-10-19T10:37:25.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:37:25.778 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:37:25.837 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.834+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f878c073990 con 0x7f878c15c950 2023-10-19T10:37:25.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.836+0000 7f8789ffb640 1 -- 172.21.15.50:0/2391928053 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f877c056f60 con 0x7f878c15c950 2023-10-19T10:37:25.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:25.839 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:25.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f877806a050 msgr2=0x7f877806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f877806a050 0x7f877806c510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8784036000 tx=0x7f8784008030 comp rx=0 tx=0).stop 2023-10-19T10:37:25.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 msgr2=0x7f878c15fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:25.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 0x7f878c15fa20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c700 tx=0x7f877c00cbc0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 shutdown_connections 2023-10-19T10:37:25.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f877806a050 0x7f877806c510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f878c15c950 0x7f878c15fa20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f878c15c570 0x7f878c15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 --2- 172.21.15.50:0/2391928053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f878c0cd920 0x7f878c1f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:25.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 >> 172.21.15.50:0/2391928053 conn(0x7f878c089ff0 msgr2=0x7f878c0cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:25.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 shutdown_connections 2023-10-19T10:37:25.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.838+0000 7f87939a0640 1 -- 172.21.15.50:0/2391928053 wait complete. 2023-10-19T10:37:25.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:25.852 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.998+0000 7f9f49c09640 1 Processor -- start 2023-10-19T10:37:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.998+0000 7f9f49c09640 1 -- start start 2023-10-19T10:37:26.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.998+0000 7f9f49c09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 0x7f9f440d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.998+0000 7f9f49c09640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440af250 con 0x7f9f44161210 2023-10-19T10:37:26.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.998+0000 7f9f49c09640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440af510 con 0x7f9f440b3090 2023-10-19T10:37:26.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.998+0000 7f9f49c09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440b0190 con 0x7f9f440b38f0 2023-10-19T10:37:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f43fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f9f44161210 0x7f9f440b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33654/0 (socket says 172.21.15.50:33654) 2023-10-19T10:37:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f43fff640 1 -- 172.21.15.50:0/887996311 learned_addr learned my addr 172.21.15.50:0/887996311 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f437fe640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 0x7f9f440d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020621053 0 0) 0x7f9f440af250 con 0x7f9f44161210 2023-10-19T10:37:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c003520 con 0x7f9f44161210 2023-10-19T10:37:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030134965 0 0) 0x7f9f2c003520 con 0x7f9f44161210 2023-10-19T10:37:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 msgr2=0x7f9f440d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 0x7f9f440d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 >> v1:172.21.15.183:6789/0 conn(0x7f9f440b3090 legacy=0x7f9f44160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440af7f0 con 0x7f9f44161210 2023-10-19T10:37:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f9f3c0019f0 con 0x7f9f44161210 2023-10-19T10:37:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 >> v1:172.21.15.50:6789/0 conn(0x7f9f44161210 legacy=0x7f9f440b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 0x7f9f2c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c003f20 0x7f9f2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c004d00 0x7f9f2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440b0190 con 0x7f9f2c003f20 2023-10-19T10:37:26.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c00ba60 con 0x7f9f2c004d00 2023-10-19T10:37:26.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c00bc20 con 0x7f9f440b38f0 2023-10-19T10:37:26.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c003f20 0x7f9f2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f437fe640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 0x7f9f2c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:25.999+0000 7f9f43fff640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c004d00 0x7f9f2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f42ffd640 1 -- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 msgr2=0x7f9f2c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:26.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f440b38f0 0x7f9f2c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f42ffd640 1 -- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c004d00 msgr2=0x7f9f2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c004d00 0x7f9f2c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f42ffd640 1 -- 172.21.15.50:0/887996311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c003520 con 0x7f9f2c003f20 2023-10-19T10:37:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c003f20 0x7f9f2c0047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f38001720 tx=0x7f9f2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1194e79048d818 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380052d0 con 0x7f9f2c003f20 2023-10-19T10:37:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3802e050 con 0x7f9f2c003f20 2023-10-19T10:37:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.000+0000 7f9f427fc640 1 -- 172.21.15.50:0/887996311 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38003830 con 0x7f9f2c003f20 2023-10-19T10:37:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f49c09640 1 -- 172.21.15.50:0/887996311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c003f20 msgr2=0x7f9f2c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f49c09640 1 --2- 172.21.15.50:0/887996311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c003f20 0x7f9f2c0047c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f38001720 tx=0x7f9f2c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f43fff640 1 -- 172.21.15.50:0/887996311 reap_dead start 2023-10-19T10:37:26.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f49c09640 1 -- 172.21.15.50:0/887996311 shutdown_connections 2023-10-19T10:37:26.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f49c09640 1 -- 172.21.15.50:0/887996311 >> 172.21.15.50:0/887996311 conn(0x7f9f4409b960 msgr2=0x7f9f440d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f49c09640 1 -- 172.21.15.50:0/887996311 shutdown_connections 2023-10-19T10:37:26.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.001+0000 7f9f49c09640 1 -- 172.21.15.50:0/887996311 wait complete. 2023-10-19T10:37:26.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.002+0000 7f9f49c09640 1 Processor -- start 2023-10-19T10:37:26.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.002+0000 7f9f49c09640 1 -- start start 2023-10-19T10:37:26.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f49c09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c003f20 0x7f9f44152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f49c09640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 0x7f9f4409c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f49c09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f4409ca60 0x7f9f440b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f49c09640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44153420 con 0x7f9f4409ebe0 2023-10-19T10:37:26.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f49c09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4409f0c0 con 0x7f9f2c003f20 2023-10-19T10:37:26.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f49c09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4409f3a0 con 0x7f9f4409ca60 2023-10-19T10:37:26.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 0x7f9f4409c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 0x7f9f4409c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39306/0 (socket says 172.21.15.50:39306) 2023-10-19T10:37:26.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 -- 172.21.15.50:0/889788294 learned_addr learned my addr 172.21.15.50:0/889788294 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f437fe640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c003f20 0x7f9f44152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f43fff640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f4409ca60 0x7f9f440b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 -- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f4409ca60 msgr2=0x7f9f440b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f4409ca60 0x7f9f440b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 -- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c003f20 msgr2=0x7f9f44152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c003f20 0x7f9f44152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.003+0000 7f9f42ffd640 1 -- 172.21.15.50:0/889788294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4409f680 con 0x7f9f4409ebe0 2023-10-19T10:37:26.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.004+0000 7f9f42ffd640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 0x7f9f4409c520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f38008fd0 tx=0x7f9f38001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.004+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380076c0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.004+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38007a30 con 0x7f9f4409ebe0 2023-10-19T10:37:26.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.004+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38006be0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.004+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4409f960 con 0x7f9f4409ebe0 2023-10-19T10:37:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.004+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f440b5e60 con 0x7f9f4409ebe0 2023-10-19T10:37:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.005+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4409f4e0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.005+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f38006d40 con 0x7f9f4409ebe0 2023-10-19T10:37:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.006+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f3006a050 0x7f9f3006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.006+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9f380aa9d0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.006+0000 7f9f437fe640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f3006a050 0x7f9f3006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.007+0000 7f9f437fe640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f3006a050 0x7f9f3006c510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f340099f0 tx=0x7f9f34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.010+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9f4409f4e0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.184+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f4409f4e0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.184+0000 7f9f2bfff640 1 -- 172.21.15.50:0/889788294 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f4409f4e0 con 0x7f9f4409ebe0 2023-10-19T10:37:26.188 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:26.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f3006a050 msgr2=0x7f9f3006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f3006a050 0x7f9f3006c510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f340099f0 tx=0x7f9f34008040 comp rx=0 tx=0).stop 2023-10-19T10:37:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 msgr2=0x7f9f4409c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 0x7f9f4409c520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f38008fd0 tx=0x7f9f38001120 comp rx=0 tx=0).stop 2023-10-19T10:37:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 shutdown_connections 2023-10-19T10:37:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f3006a050 0x7f9f3006c510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f4409ca60 0x7f9f440b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f4409ebe0 0x7f9f4409c520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 --2- 172.21.15.50:0/889788294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f2c003f20 0x7f9f44152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.186+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 >> 172.21.15.50:0/889788294 conn(0x7f9f4409b960 msgr2=0x7f9f4406b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.187+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 shutdown_connections 2023-10-19T10:37:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.187+0000 7f9f49c09640 1 -- 172.21.15.50:0/889788294 wait complete. 2023-10-19T10:37:26.200 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:26.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.307+0000 7fd38fc6a640 1 Processor -- start 2023-10-19T10:37:26.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.307+0000 7fd38fc6a640 1 -- start start 2023-10-19T10:37:26.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38fc6a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd3880b3090 0x7fd3880d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38fc6a640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880af250 con 0x7fd3880b38f0 2023-10-19T10:37:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38fc6a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880af510 con 0x7fd388161210 2023-10-19T10:37:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38fc6a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3880b0190 con 0x7fd3880b3090 2023-10-19T10:37:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38d205640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd3880b3090 0x7fd3880d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38d205640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd3880b3090 0x7fd3880d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39320/0 (socket says 172.21.15.50:39320) 2023-10-19T10:37:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38d205640 1 -- 172.21.15.50:0/3241605760 learned_addr learned my addr 172.21.15.50:0/3241605760 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38da06640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd3880b38f0 0x7fd388160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33662/0 (socket says 172.21.15.50:33662) 2023-10-19T10:37:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38d205640 1 -- 172.21.15.50:0/3241605760 >> v1:172.21.15.183:6789/0 conn(0x7fd388161210 legacy=0x7fd3880b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38d205640 1 -- 172.21.15.50:0/3241605760 >> v1:172.21.15.50:6789/0 conn(0x7fd3880b38f0 legacy=0x7fd388160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.308+0000 7fd38d205640 1 -- 172.21.15.50:0/3241605760 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880af7f0 con 0x7fd3880b3090 2023-10-19T10:37:26.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.309+0000 7fd38d205640 1 --2- 172.21.15.50:0/3241605760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880d30e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd378009870 tx=0x7fd37802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d40f9e0950590 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.309+0000 7fd38ca04640 1 -- 172.21.15.50:0/3241605760 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37802f840 con 0x7fd3880b3090 2023-10-19T10:37:26.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.309+0000 7fd38ca04640 1 -- 172.21.15.50:0/3241605760 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37802f9a0 con 0x7fd3880b3090 2023-10-19T10:37:26.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.309+0000 7fd38ca04640 1 -- 172.21.15.50:0/3241605760 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37802fd30 con 0x7fd3880b3090 2023-10-19T10:37:26.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3241605760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 msgr2=0x7fd3880d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3241605760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880d30e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd378009870 tx=0x7fd37802ba20 comp rx=0 tx=0).stop 2023-10-19T10:37:26.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3241605760 shutdown_connections 2023-10-19T10:37:26.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3241605760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880d30e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3241605760 >> 172.21.15.50:0/3241605760 conn(0x7fd38809b960 msgr2=0x7fd3880d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3241605760 shutdown_connections 2023-10-19T10:37:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.310+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3241605760 wait complete. 2023-10-19T10:37:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.311+0000 7fd38fc6a640 1 Processor -- start 2023-10-19T10:37:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.311+0000 7fd38fc6a640 1 -- start start 2023-10-19T10:37:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38fc6a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38fc6a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3880b38f0 0x7fd388150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38fc6a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd388161210 0x7fd388151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38fc6a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880cd4e0 con 0x7fd3880b3090 2023-10-19T10:37:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38fc6a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388156090 con 0x7fd388161210 2023-10-19T10:37:26.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38fc6a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880cc120 con 0x7fd3880b38f0 2023-10-19T10:37:26.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39328/0 (socket says 172.21.15.50:39328) 2023-10-19T10:37:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 -- 172.21.15.50:0/3011667595 learned_addr learned my addr 172.21.15.50:0/3011667595 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38e207640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd388161210 0x7fd388151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38d205640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3880b38f0 0x7fd388150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 -- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3880b38f0 msgr2=0x7fd388150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3880b38f0 0x7fd388150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 -- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd388161210 msgr2=0x7fd388151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd388161210 0x7fd388151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.312+0000 7fd38da06640 1 -- 172.21.15.50:0/3011667595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780094d0 con 0x7fd3880b3090 2023-10-19T10:37:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.313+0000 7fd38da06640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880cf930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c500 tx=0x7fd38400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.313+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384007500 con 0x7fd3880b3090 2023-10-19T10:37:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.313+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3840057a0 con 0x7fd3880b3090 2023-10-19T10:37:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.313+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3880cbb60 con 0x7fd3880b3090 2023-10-19T10:37:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.313+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384007660 con 0x7fd3880b3090 2023-10-19T10:37:26.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.313+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3880cd100 con 0x7fd3880b3090 2023-10-19T10:37:26.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.314+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3880ccf60 con 0x7fd3880b3090 2023-10-19T10:37:26.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.314+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd384005bd0 con 0x7fd3880b3090 2023-10-19T10:37:26.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.314+0000 7fd37e7fc640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd36c06a050 0x7fd36c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.315+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd384087de0 con 0x7fd3880b3090 2023-10-19T10:37:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.315+0000 7fd38d205640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd36c06a050 0x7fd36c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.315+0000 7fd38d205640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd36c06a050 0x7fd36c06c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd378036000 tx=0x7fd378008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.319+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd3880ccf60 con 0x7fd3880b3090 2023-10-19T10:37:26.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.495+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3880ccf60 con 0x7fd3880b3090 2023-10-19T10:37:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.496+0000 7fd37e7fc640 1 -- 172.21.15.50:0/3011667595 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd3880ccf60 con 0x7fd3880b3090 2023-10-19T10:37:26.499 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd36c06a050 msgr2=0x7fd36c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd36c06a050 0x7fd36c06c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd378036000 tx=0x7fd378008030 comp rx=0 tx=0).stop 2023-10-19T10:37:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 msgr2=0x7fd3880cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880cf930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c500 tx=0x7fd38400c9c0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 shutdown_connections 2023-10-19T10:37:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd36c06a050 0x7fd36c06c510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd388161210 0x7fd388151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3880b38f0 0x7fd388150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 --2- 172.21.15.50:0/3011667595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3880b3090 0x7fd3880cf930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 >> 172.21.15.50:0/3011667595 conn(0x7fd38809b960 msgr2=0x7fd3880c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 shutdown_connections 2023-10-19T10:37:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.498+0000 7fd38fc6a640 1 -- 172.21.15.50:0/3011667595 wait complete. 2023-10-19T10:37:26.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:26.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.617+0000 7f1e89a2f640 1 Processor -- start 2023-10-19T10:37:26.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.618+0000 7f1e89a2f640 1 -- start start 2023-10-19T10:37:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.618+0000 7f1e89a2f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 0x7f1e840b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.618+0000 7f1e89a2f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 0x7f1e840d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.618+0000 7f1e89a2f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e840b4990 con 0x7f1e8415cdf0 2023-10-19T10:37:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.618+0000 7f1e89a2f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8415d280 con 0x7f1e840b4070 2023-10-19T10:37:26.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.618+0000 7f1e89a2f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8415d560 con 0x7f1e8415c960 2023-10-19T10:37:26.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 0x7f1e840d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e82ffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 0x7f1e840b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 0x7f1e840d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:43672/0 (socket says 172.21.15.50:43672) 2023-10-19T10:37:26.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 -- 172.21.15.50:0/733915366 learned_addr learned my addr 172.21.15.50:0/733915366 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e82ffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 0x7f1e840b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:56294/0 (socket says 172.21.15.50:56294) 2023-10-19T10:37:26.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e837fe640 1 --1- 172.21.15.50:0/733915366 >> v1:172.21.15.183:6789/0 conn(0x7f1e8415cdf0 0x7f1e84160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:42028/0 (socket says 172.21.15.50:42028) 2023-10-19T10:37:26.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 -- 172.21.15.50:0/733915366 >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 msgr2=0x7f1e840b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:26.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 --2- 172.21.15.50:0/733915366 >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 0x7f1e840b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 -- 172.21.15.50:0/733915366 >> v1:172.21.15.183:6789/0 conn(0x7f1e8415cdf0 legacy=0x7f1e84160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e827fc640 1 -- 172.21.15.50:0/733915366 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8415d840 con 0x7f1e8415c960 2023-10-19T10:37:26.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.619+0000 7f1e82ffd640 1 --2- 172.21.15.50:0/733915366 >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 0x7f1e840b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e827fc640 1 --2- 172.21.15.50:0/733915366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 0x7f1e840d2d70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009870 tx=0x7f1e7802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e5a311ecd5ad11 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e81ffb640 1 -- 172.21.15.50:0/733915366 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78030840 con 0x7f1e8415c960 2023-10-19T10:37:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e81ffb640 1 -- 172.21.15.50:0/733915366 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e780309a0 con 0x7f1e8415c960 2023-10-19T10:37:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e81ffb640 1 -- 172.21.15.50:0/733915366 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78030d30 con 0x7f1e8415c960 2023-10-19T10:37:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e89a2f640 1 -- 172.21.15.50:0/733915366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 msgr2=0x7f1e840d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/733915366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 0x7f1e840d2d70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009870 tx=0x7f1e7802b980 comp rx=0 tx=0).stop 2023-10-19T10:37:26.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e89a2f640 1 -- 172.21.15.50:0/733915366 shutdown_connections 2023-10-19T10:37:26.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/733915366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e8415c960 0x7f1e840d2d70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/733915366 >> v2:172.21.15.183:3300/0 conn(0x7f1e840b4070 0x7f1e840b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.620+0000 7f1e89a2f640 1 -- 172.21.15.50:0/733915366 >> 172.21.15.50:0/733915366 conn(0x7f1e8409b970 msgr2=0x7f1e840d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.621+0000 7f1e89a2f640 1 -- 172.21.15.50:0/733915366 shutdown_connections 2023-10-19T10:37:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.621+0000 7f1e89a2f640 1 -- 172.21.15.50:0/733915366 wait complete. 2023-10-19T10:37:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.622+0000 7f1e89a2f640 1 Processor -- start 2023-10-19T10:37:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.622+0000 7f1e89a2f640 1 -- start start 2023-10-19T10:37:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e89a2f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e840b4070 0x7f1e840b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e89a2f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 0x7f1e841f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e89a2f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e840b68d0 0x7f1e841f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e89a2f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840b7b10 con 0x7f1e8415c960 2023-10-19T10:37:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e89a2f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e840cc1f0 con 0x7f1e840b68d0 2023-10-19T10:37:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e89a2f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e841f2df0 con 0x7f1e840b4070 2023-10-19T10:37:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 0x7f1e841f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 0x7f1e841f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39344/0 (socket says 172.21.15.50:39344) 2023-10-19T10:37:26.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 -- 172.21.15.50:0/2172912483 learned_addr learned my addr 172.21.15.50:0/2172912483 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e82ffd640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e840b4070 0x7f1e840b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e837fe640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e840b68d0 0x7f1e841f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 -- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e840b4070 msgr2=0x7f1e840b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e840b4070 0x7f1e840b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 -- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e840b68d0 msgr2=0x7f1e841f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e840b68d0 0x7f1e841f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 -- 172.21.15.50:0/2172912483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740093a0 con 0x7f1e8415c960 2023-10-19T10:37:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.623+0000 7f1e827fc640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 0x7f1e841f28b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000c00 tx=0x7f1e78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.624+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78039c90 con 0x7f1e8415c960 2023-10-19T10:37:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.624+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780094d0 con 0x7f1e8415c960 2023-10-19T10:37:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.624+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78039df0 con 0x7f1e8415c960 2023-10-19T10:37:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.624+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78043640 con 0x7f1e8415c960 2023-10-19T10:37:26.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.624+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e840b7170 con 0x7f1e8415c960 2023-10-19T10:37:26.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.624+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e840b7c50 con 0x7f1e8415c960 2023-10-19T10:37:26.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.625+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e780437a0 con 0x7f1e8415c960 2023-10-19T10:37:26.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.625+0000 7f1e6bfff640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1e7006a050 0x7f1e7006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.625+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1e7803e070 con 0x7f1e8415c960 2023-10-19T10:37:26.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.626+0000 7f1e82ffd640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1e7006a050 0x7f1e7006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.626+0000 7f1e82ffd640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1e7006a050 0x7f1e7006c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e8414ff30 tx=0x7f1e74009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.630+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1e840b7c50 con 0x7f1e8415c960 2023-10-19T10:37:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.804+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e840b7c50 con 0x7f1e8415c960 2023-10-19T10:37:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.804+0000 7f1e6bfff640 1 -- 172.21.15.50:0/2172912483 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e840b7c50 con 0x7f1e8415c960 2023-10-19T10:37:26.807 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:26.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1e7006a050 msgr2=0x7f1e7006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1e7006a050 0x7f1e7006c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e8414ff30 tx=0x7f1e74009110 comp rx=0 tx=0).stop 2023-10-19T10:37:26.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 msgr2=0x7f1e841f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 0x7f1e841f28b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000c00 tx=0x7f1e78004730 comp rx=0 tx=0).stop 2023-10-19T10:37:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 shutdown_connections 2023-10-19T10:37:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1e7006a050 0x7f1e7006c510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e840b68d0 0x7f1e841f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e8415c960 0x7f1e841f28b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 --2- 172.21.15.50:0/2172912483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e840b4070 0x7f1e840b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 >> 172.21.15.50:0/2172912483 conn(0x7f1e8409b970 msgr2=0x7f1e8415e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 shutdown_connections 2023-10-19T10:37:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.806+0000 7f1e89a2f640 1 -- 172.21.15.50:0/2172912483 wait complete. 2023-10-19T10:37:26.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:26.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.926+0000 7f42d8b03640 1 Processor -- start 2023-10-19T10:37:26.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.926+0000 7f42d8b03640 1 -- start start 2023-10-19T10:37:26.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d8b03640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f42d40b4070 0x7f42d40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d8b03640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42d415cdf0 0x7f42d40cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d8b03640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d40d3480 con 0x7f42d415c960 2023-10-19T10:37:26.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d8b03640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d40b4990 con 0x7f42d40b4070 2023-10-19T10:37:26.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d8b03640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d415d280 con 0x7f42d415cdf0 2023-10-19T10:37:26.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f42d40b4070 0x7f42d40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f42d40b4070 0x7f42d40b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38388/0 (socket says 172.21.15.50:38388) 2023-10-19T10:37:26.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 -- 172.21.15.50:0/3475984020 learned_addr learned my addr 172.21.15.50:0/3475984020 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d2d9d640 1 --2- 172.21.15.50:0/3475984020 >> v2:172.21.15.183:3300/0 conn(0x7f42d415cdf0 0x7f42d40cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 -- 172.21.15.50:0/3475984020 >> v2:172.21.15.183:3300/0 conn(0x7f42d415cdf0 msgr2=0x7f42d40cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:26.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 --2- 172.21.15.50:0/3475984020 >> v2:172.21.15.183:3300/0 conn(0x7f42d415cdf0 0x7f42d40cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 -- 172.21.15.50:0/3475984020 >> v1:172.21.15.183:6789/0 conn(0x7f42d415c960 legacy=0x7f42d40d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.927+0000 7f42d259c640 1 -- 172.21.15.50:0/3475984020 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d415d560 con 0x7f42d40b4070 2023-10-19T10:37:26.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.928+0000 7f42d259c640 1 --2- 172.21.15.50:0/3475984020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d40b4070 0x7f42d40b4450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42d40c6030 tx=0x7f42cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d470e56cae0c4143 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.928+0000 7f42d159a640 1 -- 172.21.15.50:0/3475984020 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc004230 con 0x7f42d40b4070 2023-10-19T10:37:26.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.928+0000 7f42d159a640 1 -- 172.21.15.50:0/3475984020 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc004390 con 0x7f42d40b4070 2023-10-19T10:37:26.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.928+0000 7f42d159a640 1 -- 172.21.15.50:0/3475984020 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc030840 con 0x7f42d40b4070 2023-10-19T10:37:26.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 -- 172.21.15.50:0/3475984020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d40b4070 msgr2=0x7f42d40b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 --2- 172.21.15.50:0/3475984020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d40b4070 0x7f42d40b4450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42d40c6030 tx=0x7f42cc02b980 comp rx=0 tx=0).stop 2023-10-19T10:37:26.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 -- 172.21.15.50:0/3475984020 shutdown_connections 2023-10-19T10:37:26.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 --2- 172.21.15.50:0/3475984020 >> v2:172.21.15.183:3300/0 conn(0x7f42d415cdf0 0x7f42d40cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 --2- 172.21.15.50:0/3475984020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d40b4070 0x7f42d40b4450 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 -- 172.21.15.50:0/3475984020 >> 172.21.15.50:0/3475984020 conn(0x7f42d409b970 msgr2=0x7f42d40d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:26.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 -- 172.21.15.50:0/3475984020 shutdown_connections 2023-10-19T10:37:26.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.929+0000 7f42d8b03640 1 -- 172.21.15.50:0/3475984020 wait complete. 2023-10-19T10:37:26.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.930+0000 7f42d8b03640 1 Processor -- start 2023-10-19T10:37:26.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.930+0000 7f42d8b03640 1 -- start start 2023-10-19T10:37:26.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d8b03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d40b4070 0x7f42d4158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d8b03640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 0x7f42d40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d8b03640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42d40cbb10 0x7f42d40c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d8b03640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c92d0 con 0x7f42d415c960 2023-10-19T10:37:26.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d8b03640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40ca810 con 0x7f42d40b4070 2023-10-19T10:37:26.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d8b03640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40caaf0 con 0x7f42d40cbb10 2023-10-19T10:37:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 0x7f42d40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 0x7f42d40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38394/0 (socket says 172.21.15.50:38394) 2023-10-19T10:37:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 -- 172.21.15.50:0/2239410965 learned_addr learned my addr 172.21.15.50:0/2239410965 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d2d9d640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42d40cbb10 0x7f42d40c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d259c640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d40b4070 0x7f42d4158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 -- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42d40cbb10 msgr2=0x7f42d40c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42d40cbb10 0x7f42d40c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 -- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d40b4070 msgr2=0x7f42d4158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:26.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d40b4070 0x7f42d4158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:26.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d1d9b640 1 -- 172.21.15.50:0/2239410965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc009460 con 0x7f42d415c960 2023-10-19T10:37:26.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.931+0000 7f42d2d9d640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42d40cbb10 0x7f42d40c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:26.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.932+0000 7f42d1d9b640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 0x7f42d40cb5d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c450 tx=0x7f42bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.932+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0052d0 con 0x7f42d415c960 2023-10-19T10:37:26.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.932+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc005430 con 0x7f42d415c960 2023-10-19T10:37:26.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.932+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d40cadd0 con 0x7f42d415c960 2023-10-19T10:37:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.932+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc012870 con 0x7f42d415c960 2023-10-19T10:37:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.932+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d41591d0 con 0x7f42d415c960 2023-10-19T10:37:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.933+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d404e420 con 0x7f42d415c960 2023-10-19T10:37:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.933+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f42bc0129d0 con 0x7f42d415c960 2023-10-19T10:37:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.934+0000 7f42caffd640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f42b406a050 0x7f42b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.934+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f42bc0883d0 con 0x7f42d415c960 2023-10-19T10:37:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.934+0000 7f42d259c640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f42b406a050 0x7f42b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.934+0000 7f42d259c640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f42b406a050 0x7f42b406c510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f42cc035000 tx=0x7f42cc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:26.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:26.938+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f42bc0537d0 con 0x7f42d415c960 2023-10-19T10:37:27.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.114+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42d408c720 con 0x7f42d415c960 2023-10-19T10:37:27.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.115+0000 7f42caffd640 1 -- 172.21.15.50:0/2239410965 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42bc056df0 con 0x7f42d415c960 2023-10-19T10:37:27.118 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:27.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f42b406a050 msgr2=0x7f42b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f42b406a050 0x7f42b406c510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f42cc035000 tx=0x7f42cc008030 comp rx=0 tx=0).stop 2023-10-19T10:37:27.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 msgr2=0x7f42d40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 0x7f42d40cb5d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c450 tx=0x7f42bc00c910 comp rx=0 tx=0).stop 2023-10-19T10:37:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 shutdown_connections 2023-10-19T10:37:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f42b406a050 0x7f42b406c510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42d40cbb10 0x7f42d40c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42d415c960 0x7f42d40cb5d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 --2- 172.21.15.50:0/2239410965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d40b4070 0x7f42d4158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 >> 172.21.15.50:0/2239410965 conn(0x7f42d409b970 msgr2=0x7f42d415e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 shutdown_connections 2023-10-19T10:37:27.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.117+0000 7f42d8b03640 1 -- 172.21.15.50:0/2239410965 wait complete. 2023-10-19T10:37:27.130 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.237+0000 7fe793a84640 1 Processor -- start 2023-10-19T10:37:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.237+0000 7fe793a84640 1 -- start start 2023-10-19T10:37:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe793a84640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe78c0af3c0 0x7fe78c0af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe793a84640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe78c0ca600 0x7fe78c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe793a84640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78c0aec30 con 0x7fe78c0ad410 2023-10-19T10:37:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe793a84640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c0afd80 con 0x7fe78c0ca600 2023-10-19T10:37:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe793a84640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c0add80 con 0x7fe78c0af3c0 2023-10-19T10:37:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe792021640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe78c0ca600 0x7fe78c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe792021640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe78c0ca600 0x7fe78c0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38396/0 (socket says 172.21.15.50:38396) 2023-10-19T10:37:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe792021640 1 -- 172.21.15.50:0/2377721 learned_addr learned my addr 172.21.15.50:0/2377721 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.238+0000 7fe79101f640 1 --2- 172.21.15.50:0/2377721 >> v2:172.21.15.183:3300/0 conn(0x7fe78c0af3c0 0x7fe78c0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.239+0000 7fe792021640 1 -- 172.21.15.50:0/2377721 >> v2:172.21.15.183:3300/0 conn(0x7fe78c0af3c0 msgr2=0x7fe78c0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.239+0000 7fe792021640 1 --2- 172.21.15.50:0/2377721 >> v2:172.21.15.183:3300/0 conn(0x7fe78c0af3c0 0x7fe78c0af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.239+0000 7fe792021640 1 -- 172.21.15.50:0/2377721 >> v1:172.21.15.183:6789/0 conn(0x7fe78c0ad410 legacy=0x7fe78c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.239+0000 7fe792021640 1 -- 172.21.15.50:0/2377721 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c0ae020 con 0x7fe78c0ca600 2023-10-19T10:37:27.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.239+0000 7fe792021640 1 --2- 172.21.15.50:0/2377721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0caa00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe78c06e150 tx=0x7fe78402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8afb01869e8ceed server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.239+0000 7fe79081e640 1 -- 172.21.15.50:0/2377721 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784030860 con 0x7fe78c0ca600 2023-10-19T10:37:27.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe79081e640 1 -- 172.21.15.50:0/2377721 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7840309c0 con 0x7fe78c0ca600 2023-10-19T10:37:27.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe79081e640 1 -- 172.21.15.50:0/2377721 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784030d50 con 0x7fe78c0ca600 2023-10-19T10:37:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 -- 172.21.15.50:0/2377721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 msgr2=0x7fe78c0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 --2- 172.21.15.50:0/2377721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0caa00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe78c06e150 tx=0x7fe78402b9a0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 -- 172.21.15.50:0/2377721 shutdown_connections 2023-10-19T10:37:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 --2- 172.21.15.50:0/2377721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0caa00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 --2- 172.21.15.50:0/2377721 >> v2:172.21.15.183:3300/0 conn(0x7fe78c0af3c0 0x7fe78c0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 -- 172.21.15.50:0/2377721 >> 172.21.15.50:0/2377721 conn(0x7fe78c070e70 msgr2=0x7fe78c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 -- 172.21.15.50:0/2377721 shutdown_connections 2023-10-19T10:37:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.240+0000 7fe793a84640 1 -- 172.21.15.50:0/2377721 wait complete. 2023-10-19T10:37:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.241+0000 7fe793a84640 1 Processor -- start 2023-10-19T10:37:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.241+0000 7fe793a84640 1 -- start start 2023-10-19T10:37:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe793a84640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe78c0ad410 0x7fe78c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe793a84640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe78c0af3c0 0x7fe78c0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe793a84640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe793a84640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c0ccfe0 con 0x7fe78c0ca600 2023-10-19T10:37:27.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe793a84640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c0cc080 con 0x7fe78c0ad410 2023-10-19T10:37:27.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe793a84640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c0cc360 con 0x7fe78c0af3c0 2023-10-19T10:37:27.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe792021640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe792021640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38408/0 (socket says 172.21.15.50:38408) 2023-10-19T10:37:27.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe792021640 1 -- 172.21.15.50:0/2380835599 learned_addr learned my addr 172.21.15.50:0/2380835599 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.242+0000 7fe791820640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe78c0ad410 0x7fe78c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe792021640 1 -- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe78c0af3c0 msgr2=0x7fe78c0cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe79101f640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe78c0af3c0 0x7fe78c0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe792021640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe78c0af3c0 0x7fe78c0cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe792021640 1 -- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe78c0ad410 msgr2=0x7fe78c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe792021640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe78c0ad410 0x7fe78c154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe792021640 1 -- 172.21.15.50:0/2380835599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840094d0 con 0x7fe78c0ca600 2023-10-19T10:37:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe791820640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe78c0ad410 0x7fe78c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe792021640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0cdd40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe78402b9e0 tx=0x7fe784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784039c90 con 0x7fe78c0ca600 2023-10-19T10:37:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784039e10 con 0x7fe78c0ca600 2023-10-19T10:37:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c0cc640 con 0x7fe78c0ca600 2023-10-19T10:37:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784043640 con 0x7fe78c0ca600 2023-10-19T10:37:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.243+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c0ccc00 con 0x7fe78c0ca600 2023-10-19T10:37:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.244+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c0c6a50 con 0x7fe78c0ca600 2023-10-19T10:37:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.244+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe7840437a0 con 0x7fe78c0ca600 2023-10-19T10:37:27.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.245+0000 7fe78a7fc640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe77806a050 0x7fe77806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.245+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe78403e070 con 0x7fe78c0ca600 2023-10-19T10:37:27.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.245+0000 7fe791820640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe77806a050 0x7fe77806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.246+0000 7fe791820640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe77806a050 0x7fe77806c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe77c001cd0 tx=0x7fe77c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.249+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe784076e30 con 0x7fe78c0ca600 2023-10-19T10:37:27.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.424+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe78c04e420 con 0x7fe78c0ca600 2023-10-19T10:37:27.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.425+0000 7fe78a7fc640 1 -- 172.21.15.50:0/2380835599 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe78407a450 con 0x7fe78c0ca600 2023-10-19T10:37:27.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe77806a050 msgr2=0x7fe77806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe77806a050 0x7fe77806c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe77c001cd0 tx=0x7fe77c009040 comp rx=0 tx=0).stop 2023-10-19T10:37:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 msgr2=0x7fe78c0cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0cdd40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe78402b9e0 tx=0x7fe784004730 comp rx=0 tx=0).stop 2023-10-19T10:37:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 shutdown_connections 2023-10-19T10:37:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe77806a050 0x7fe77806c510 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c0ca600 0x7fe78c0cdd40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe78c0af3c0 0x7fe78c0cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 --2- 172.21.15.50:0/2380835599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe78c0ad410 0x7fe78c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 >> 172.21.15.50:0/2380835599 conn(0x7fe78c070e70 msgr2=0x7fe78c0c4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:27.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 shutdown_connections 2023-10-19T10:37:27.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.427+0000 7fe793a84640 1 -- 172.21.15.50:0/2380835599 wait complete. 2023-10-19T10:37:27.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:37:27.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.544+0000 7f93b2802640 1 Processor -- start 2023-10-19T10:37:27.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.544+0000 7f93b2802640 1 -- start start 2023-10-19T10:37:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b2802640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93ac0cd300 0x7f93ac15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b2802640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f93ac0b7fc0 0x7f93ac15c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b2802640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac0b6de0 con 0x7f93ac0cd6e0 2023-10-19T10:37:27.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b2802640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac0b7960 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b2802640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac0b70c0 con 0x7f93ac0cd300 2023-10-19T10:37:27.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b0d9f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f93ac0b7fc0 0x7f93ac15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93ab7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f93ac0cd6e0 0x7f93ac0d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41264/0 (socket says 172.21.15.50:41264) 2023-10-19T10:37:27.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b0d9f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f93ac0b7fc0 0x7f93ac15c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38418/0 (socket says 172.21.15.50:38418) 2023-10-19T10:37:27.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93ab7fe640 1 -- 172.21.15.50:0/1307279197 learned_addr learned my addr 172.21.15.50:0/1307279197 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:27.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93abfff640 1 --2- 172.21.15.50:0/1307279197 >> v2:172.21.15.183:3300/0 conn(0x7f93ac0cd300 0x7f93ac15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b0d9f640 1 -- 172.21.15.50:0/1307279197 >> v2:172.21.15.183:3300/0 conn(0x7f93ac0cd300 msgr2=0x7f93ac15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:27.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b0d9f640 1 --2- 172.21.15.50:0/1307279197 >> v2:172.21.15.183:3300/0 conn(0x7f93ac0cd300 0x7f93ac15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.545+0000 7f93b0d9f640 1 -- 172.21.15.50:0/1307279197 >> v1:172.21.15.50:6789/0 conn(0x7f93ac0cd6e0 legacy=0x7f93ac0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93aaffd640 1 -- 172.21.15.50:0/1307279197 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841365160 0 0) 0x7f93ac0b6de0 con 0x7f93ac0cd6e0 2023-10-19T10:37:27.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93b0d9f640 1 -- 172.21.15.50:0/1307279197 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0b73a0 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93abfff640 1 --2- 172.21.15.50:0/1307279197 >> v2:172.21.15.183:3300/0 conn(0x7f93ac0cd300 0x7f93ac15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:27.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93b0d9f640 1 --2- 172.21.15.50:0/1307279197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac15c530 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0b4d20 tx=0x7f93a402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150a9d8883aeba31 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93aaffd640 1 -- 172.21.15.50:0/1307279197 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a402cd00 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93aaffd640 1 -- 172.21.15.50:0/1307279197 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a402ce60 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.546+0000 7f93aaffd640 1 -- 172.21.15.50:0/1307279197 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4034660 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 -- 172.21.15.50:0/1307279197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 msgr2=0x7f93ac15c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 --2- 172.21.15.50:0/1307279197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac15c530 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0b4d20 tx=0x7f93a402bf50 comp rx=0 tx=0).stop 2023-10-19T10:37:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 -- 172.21.15.50:0/1307279197 shutdown_connections 2023-10-19T10:37:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 --2- 172.21.15.50:0/1307279197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac15c530 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 --2- 172.21.15.50:0/1307279197 >> v2:172.21.15.183:3300/0 conn(0x7f93ac0cd300 0x7f93ac15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 -- 172.21.15.50:0/1307279197 >> 172.21.15.50:0/1307279197 conn(0x7f93ac071320 msgr2=0x7f93ac071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 -- 172.21.15.50:0/1307279197 shutdown_connections 2023-10-19T10:37:27.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.547+0000 7f93b2802640 1 -- 172.21.15.50:0/1307279197 wait complete. 2023-10-19T10:37:27.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.548+0000 7f93b2802640 1 Processor -- start 2023-10-19T10:37:27.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.548+0000 7f93b2802640 1 -- start start 2023-10-19T10:37:27.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b2802640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b2802640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93ac0cd300 0x7f93ac0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b2802640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ac0cd6e0 0x7f93ac0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b2802640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0c9a70 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b2802640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0c9d50 con 0x7f93ac0cd6e0 2023-10-19T10:37:27.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b2802640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0ca030 con 0x7f93ac0cd300 2023-10-19T10:37:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac0c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38430/0 (socket says 172.21.15.50:38430) 2023-10-19T10:37:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 -- 172.21.15.50:0/3299374066 learned_addr learned my addr 172.21.15.50:0/3299374066 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b0d9f640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ac0cd6e0 0x7f93ac0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 -- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93ac0cd300 msgr2=0x7f93ac0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93ab7fe640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93ac0cd300 0x7f93ac0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93ac0cd300 0x7f93ac0cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 -- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ac0cd6e0 msgr2=0x7f93ac0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ac0cd6e0 0x7f93ac0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93abfff640 1 -- 172.21.15.50:0/3299374066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a4009a70 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.549+0000 7f93b0d9f640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ac0cd6e0 0x7f93ac0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:27.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.550+0000 7f93abfff640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac0c9490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f93a000e730 tx=0x7f93a000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.550+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0004230 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.550+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0004390 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.550+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac0ca310 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.550+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000c8f0 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.550+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac0c70c0 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.551+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac073990 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.551+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f93a000ca50 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.552+0000 7f93a8ff9640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f939806a050 0x7f939806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.552+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f93a00884f0 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.552+0000 7f93ab7fe640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f939806a050 0x7f939806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.552+0000 7f93ab7fe640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f939806a050 0x7f939806c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0d31e0 tx=0x7f939c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.556+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f93a00538f0 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.766+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93ac0cdeb0 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.767+0000 7f93a8ff9640 1 -- 172.21.15.50:0/3299374066 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7f93a0056f10 con 0x7f93ac0b7fc0 2023-10-19T10:37:27.770 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:27.770 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:37:27.771 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:37:27.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f939806a050 msgr2=0x7f939806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f939806a050 0x7f939806c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0d31e0 tx=0x7f939c009040 comp rx=0 tx=0).stop 2023-10-19T10:37:27.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 msgr2=0x7f93ac0c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac0c9490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f93a000e730 tx=0x7f93a000ebf0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 shutdown_connections 2023-10-19T10:37:27.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f939806a050 0x7f939806c510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ac0cd6e0 0x7f93ac0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93ac0cd300 0x7f93ac0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 --2- 172.21.15.50:0/3299374066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93ac0b7fc0 0x7f93ac0c9490 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 >> 172.21.15.50:0/3299374066 conn(0x7f93ac071320 msgr2=0x7f93ac1e6740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:27.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 shutdown_connections 2023-10-19T10:37:27.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.769+0000 7f93b2802640 1 -- 172.21.15.50:0/3299374066 wait complete. 2023-10-19T10:37:27.782 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:27.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 Processor -- start 2023-10-19T10:37:27.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 -- start start 2023-10-19T10:37:27.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fda5c156510 0x7fda5c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c159210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fda5c159750 0x7fda5c06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c06f770 con 0x7fda5c156510 2023-10-19T10:37:27.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0c95d0 con 0x7fda5c159750 2023-10-19T10:37:27.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.887+0000 7fda618e7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0cb240 con 0x7fda5c0cadc0 2023-10-19T10:37:27.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fda5c156510 0x7fda5c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fda5c156510 0x7fda5c0ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38444/0 (socket says 172.21.15.50:38444) 2023-10-19T10:37:27.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 -- 172.21.15.50:0/898923733 learned_addr learned my addr 172.21.15.50:0/898923733 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:27.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5b7fe640 1 --2- 172.21.15.50:0/898923733 >> v2:172.21.15.183:3300/0 conn(0x7fda5c159750 0x7fda5c06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda53fff640 1 --2- 172.21.15.50:0/898923733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 -- 172.21.15.50:0/898923733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 msgr2=0x7fda5c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:27.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 --2- 172.21.15.50:0/898923733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 -- 172.21.15.50:0/898923733 >> v2:172.21.15.183:3300/0 conn(0x7fda5c159750 msgr2=0x7fda5c06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:27.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 --2- 172.21.15.50:0/898923733 >> v2:172.21.15.183:3300/0 conn(0x7fda5c159750 0x7fda5c06b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.888+0000 7fda5affd640 1 -- 172.21.15.50:0/898923733 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0c8ca0 con 0x7fda5c156510 2023-10-19T10:37:27.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda5affd640 1 --2- 172.21.15.50:0/898923733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c156510 0x7fda5c0ca880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda5c06c770 tx=0x7fda4c02bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2ed3b9e5e2c7f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda5a7fc640 1 -- 172.21.15.50:0/898923733 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c004230 con 0x7fda5c156510 2023-10-19T10:37:27.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda5a7fc640 1 -- 172.21.15.50:0/898923733 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c004390 con 0x7fda5c156510 2023-10-19T10:37:27.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda5a7fc640 1 -- 172.21.15.50:0/898923733 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c02c850 con 0x7fda5c156510 2023-10-19T10:37:27.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda618e7640 1 -- 172.21.15.50:0/898923733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c156510 msgr2=0x7fda5c0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda618e7640 1 --2- 172.21.15.50:0/898923733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c156510 0x7fda5c0ca880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda5c06c770 tx=0x7fda4c02bcb0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.889+0000 7fda618e7640 1 -- 172.21.15.50:0/898923733 shutdown_connections 2023-10-19T10:37:27.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.890+0000 7fda618e7640 1 --2- 172.21.15.50:0/898923733 >> v2:172.21.15.183:3300/0 conn(0x7fda5c159750 0x7fda5c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.890+0000 7fda618e7640 1 --2- 172.21.15.50:0/898923733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.890+0000 7fda618e7640 1 --2- 172.21.15.50:0/898923733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c156510 0x7fda5c0ca880 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.890+0000 7fda618e7640 1 -- 172.21.15.50:0/898923733 >> 172.21.15.50:0/898923733 conn(0x7fda5c069e90 msgr2=0x7fda5c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:27.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.890+0000 7fda618e7640 1 -- 172.21.15.50:0/898923733 shutdown_connections 2023-10-19T10:37:27.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.890+0000 7fda618e7640 1 -- 172.21.15.50:0/898923733 wait complete. 2023-10-19T10:37:27.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 Processor -- start 2023-10-19T10:37:27.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 -- start start 2023-10-19T10:37:27.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c156510 0x7fda5c0c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 0x7fda5c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c14dcf0 con 0x7fda5c159750 2023-10-19T10:37:27.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c14df90 con 0x7fda5c156510 2023-10-19T10:37:27.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.891+0000 7fda618e7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c14e270 con 0x7fda5c0cadc0 2023-10-19T10:37:27.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 0x7fda5c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 0x7fda5c15afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38448/0 (socket says 172.21.15.50:38448) 2023-10-19T10:37:27.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 -- 172.21.15.50:0/3641629516 learned_addr learned my addr 172.21.15.50:0/3641629516 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:27.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5affd640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda53fff640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c156510 0x7fda5c0c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 -- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 msgr2=0x7fda5c14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c14d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 -- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c156510 msgr2=0x7fda5c0c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:27.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c156510 0x7fda5c0c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:27.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 -- 172.21.15.50:0/3641629516 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c00a580 con 0x7fda5c159750 2023-10-19T10:37:27.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda5b7fe640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 0x7fda5c15afd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fda5400bde0 tx=0x7fda5400e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54010040 con 0x7fda5c159750 2023-10-19T10:37:27.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5400ed20 con 0x7fda5c159750 2023-10-19T10:37:27.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c14e550 con 0x7fda5c159750 2023-10-19T10:37:27.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54003bd0 con 0x7fda5c159750 2023-10-19T10:37:27.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.892+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0acf50 con 0x7fda5c159750 2023-10-19T10:37:27.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.893+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c14ea70 con 0x7fda5c159750 2023-10-19T10:37:27.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.894+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fda54003d30 con 0x7fda5c159750 2023-10-19T10:37:27.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.894+0000 7fda58ff9640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a080 0x7fda3c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:27.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.894+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda54088a00 con 0x7fda5c159750 2023-10-19T10:37:27.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.894+0000 7fda5affd640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a080 0x7fda3c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:27.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.895+0000 7fda5affd640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a080 0x7fda3c06c540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda4c035000 tx=0x7fda4c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:27.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:27.898+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fda5c14ea70 con 0x7fda5c159750 2023-10-19T10:37:28.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.073+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda5c073990 con 0x7fda5c159750 2023-10-19T10:37:28.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.074+0000 7fda58ff9640 1 -- 172.21.15.50:0/3641629516 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7fda54057920 con 0x7fda5c159750 2023-10-19T10:37:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:28.077 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:28.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a080 msgr2=0x7fda3c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a080 0x7fda3c06c540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda4c035000 tx=0x7fda4c030000 comp rx=0 tx=0).stop 2023-10-19T10:37:28.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 msgr2=0x7fda5c15afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 0x7fda5c15afd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fda5400bde0 tx=0x7fda5400e490 comp rx=0 tx=0).stop 2023-10-19T10:37:28.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 shutdown_connections 2023-10-19T10:37:28.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a080 0x7fda3c06c540 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c159750 0x7fda5c15afd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c156510 0x7fda5c0c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 --2- 172.21.15.50:0/3641629516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0cadc0 0x7fda5c14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 >> 172.21.15.50:0/3641629516 conn(0x7fda5c069e90 msgr2=0x7fda5c151970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:28.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 shutdown_connections 2023-10-19T10:37:28.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.076+0000 7fda618e7640 1 -- 172.21.15.50:0/3641629516 wait complete. 2023-10-19T10:37:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:28.091 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:28.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.198+0000 7fdfda2dc640 1 Processor -- start 2023-10-19T10:37:28.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.198+0000 7fdfda2dc640 1 -- start start 2023-10-19T10:37:28.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.198+0000 7fdfda2dc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd40b4070 0x7fdfd40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.198+0000 7fdfda2dc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd40d3480 con 0x7fdfd415c960 2023-10-19T10:37:28.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.198+0000 7fdfda2dc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd40b4990 con 0x7fdfd415cdc0 2023-10-19T10:37:28.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.198+0000 7fdfda2dc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd415d280 con 0x7fdfd40b4070 2023-10-19T10:37:28.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd37fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fdfd415c960 0x7fdfd40d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41266/0 (socket says 172.21.15.50:41266) 2023-10-19T10:37:28.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd37fe640 1 -- 172.21.15.50:0/2265760375 learned_addr learned my addr 172.21.15.50:0/2265760375 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:28.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd3fff640 1 --2- 172.21.15.50:0/2265760375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd40b4070 0x7fdfd40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372848894 0 0) 0x7fdfd40d3480 con 0x7fdfd415c960 2023-10-19T10:37:28.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfbc003520 con 0x7fdfd415c960 2023-10-19T10:37:28.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885722087 0 0) 0x7fdfd40b4990 con 0x7fdfd415cdc0 2023-10-19T10:37:28.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd40d3480 con 0x7fdfd415cdc0 2023-10-19T10:37:28.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680261058 0 0) 0x7fdfbc003520 con 0x7fdfd415c960 2023-10-19T10:37:28.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd3fff640 1 -- 172.21.15.50:0/2265760375 >> v1:172.21.15.183:6789/0 conn(0x7fdfd415cdc0 legacy=0x7fdfd40cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd3fff640 1 -- 172.21.15.50:0/2265760375 >> v1:172.21.15.50:6789/0 conn(0x7fdfd415c960 legacy=0x7fdfd40d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.199+0000 7fdfd3fff640 1 -- 172.21.15.50:0/2265760375 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd415d560 con 0x7fdfd40b4070 2023-10-19T10:37:28.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.200+0000 7fdfd3fff640 1 --2- 172.21.15.50:0/2265760375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd40b4070 0x7fdfd40b4450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40097c0 tx=0x7fdfc402b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ad29b2b864b8e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.200+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8001690 con 0x7fdfd40b4070 2023-10-19T10:37:28.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.200+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc40314d0 con 0x7fdfd40b4070 2023-10-19T10:37:28.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.200+0000 7fdfd2ffd640 1 -- 172.21.15.50:0/2265760375 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4031840 con 0x7fdfd40b4070 2023-10-19T10:37:28.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 -- 172.21.15.50:0/2265760375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd40b4070 msgr2=0x7fdfd40b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/2265760375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd40b4070 0x7fdfd40b4450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40097c0 tx=0x7fdfc402b7f0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 -- 172.21.15.50:0/2265760375 shutdown_connections 2023-10-19T10:37:28.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/2265760375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd40b4070 0x7fdfd40b4450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 -- 172.21.15.50:0/2265760375 >> 172.21.15.50:0/2265760375 conn(0x7fdfd409b970 msgr2=0x7fdfd40d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:28.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 -- 172.21.15.50:0/2265760375 shutdown_connections 2023-10-19T10:37:28.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.201+0000 7fdfda2dc640 1 -- 172.21.15.50:0/2265760375 wait complete. 2023-10-19T10:37:28.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.202+0000 7fdfda2dc640 1 Processor -- start 2023-10-19T10:37:28.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.202+0000 7fdfda2dc640 1 -- start start 2023-10-19T10:37:28.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfda2dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 0x7fdfd4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfda2dc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd415c960 0x7fdfd40c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfda2dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd415cdc0 0x7fdfd40c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfda2dc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40c8e60 con 0x7fdfd40b4070 2023-10-19T10:37:28.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfda2dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41f6050 con 0x7fdfd415cdc0 2023-10-19T10:37:28.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfda2dc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41f5480 con 0x7fdfd415c960 2023-10-19T10:37:28.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 0x7fdfd4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 0x7fdfd4157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38460/0 (socket says 172.21.15.50:38460) 2023-10-19T10:37:28.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 -- 172.21.15.50:0/1267315574 learned_addr learned my addr 172.21.15.50:0/1267315574 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:28.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd8879640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd415cdc0 0x7fdfd40c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd37fe640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd415c960 0x7fdfd40c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 -- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd415c960 msgr2=0x7fdfd40c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd415c960 0x7fdfd40c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 -- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd415cdc0 msgr2=0x7fdfd40c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd415cdc0 0x7fdfd40c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 -- 172.21.15.50:0/1267315574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40093a0 con 0x7fdfd40b4070 2023-10-19T10:37:28.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.203+0000 7fdfd3fff640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 0x7fdfd4157df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000c00 tx=0x7fdfc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.204+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc403f950 con 0x7fdfd40b4070 2023-10-19T10:37:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.204+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc403fab0 con 0x7fdfd40b4070 2023-10-19T10:37:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.204+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc403fe60 con 0x7fdfd40b4070 2023-10-19T10:37:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.204+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd41f5720 con 0x7fdfd40b4070 2023-10-19T10:37:28.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.204+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd41f5ce0 con 0x7fdfd40b4070 2023-10-19T10:37:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.205+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd40d09b0 con 0x7fdfd40b4070 2023-10-19T10:37:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.205+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdfc403b040 con 0x7fdfd40b4070 2023-10-19T10:37:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.205+0000 7fdfd0ff9640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdfc006a080 0x7fdfc006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.206+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdfc40aa830 con 0x7fdfd40b4070 2023-10-19T10:37:28.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.206+0000 7fdfd37fe640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdfc006a080 0x7fdfc006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.206+0000 7fdfd37fe640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdfc006a080 0x7fdfc006c540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80017c0 tx=0x7fdfc8001530 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.210+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fdfc4075d60 con 0x7fdfd40b4070 2023-10-19T10:37:28.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.384+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdfd41f5e70 con 0x7fdfd40b4070 2023-10-19T10:37:28.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.384+0000 7fdfd0ff9640 1 -- 172.21.15.50:0/1267315574 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdfc4079380 con 0x7fdfd40b4070 2023-10-19T10:37:28.387 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:28.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdfc006a080 msgr2=0x7fdfc006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdfc006a080 0x7fdfc006c540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80017c0 tx=0x7fdfc8001530 comp rx=0 tx=0).stop 2023-10-19T10:37:28.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 msgr2=0x7fdfd4157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 0x7fdfd4157df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000c00 tx=0x7fdfc40046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 shutdown_connections 2023-10-19T10:37:28.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdfc006a080 0x7fdfc006c540 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd415cdc0 0x7fdfd40c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfd415c960 0x7fdfd40c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 --2- 172.21.15.50:0/1267315574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfd40b4070 0x7fdfd4157df0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 >> 172.21.15.50:0/1267315574 conn(0x7fdfd409b970 msgr2=0x7fdfd41601f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:28.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 shutdown_connections 2023-10-19T10:37:28.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.386+0000 7fdfda2dc640 1 -- 172.21.15.50:0/1267315574 wait complete. 2023-10-19T10:37:28.400 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:28.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.506+0000 7feff7beb640 1 Processor -- start 2023-10-19T10:37:28.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.506+0000 7feff7beb640 1 -- start start 2023-10-19T10:37:28.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff7beb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7feff00b3090 0x7feff00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff7beb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00af250 con 0x7feff0161210 2023-10-19T10:37:28.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff7beb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00af510 con 0x7feff00b38f0 2023-10-19T10:37:28.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff7beb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff00b0190 con 0x7feff00b3090 2023-10-19T10:37:28.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff5186640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7feff00b3090 0x7feff00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff6188640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7feff0161210 0x7feff00b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41280/0 (socket says 172.21.15.50:41280) 2023-10-19T10:37:28.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff6188640 1 -- 172.21.15.50:0/3005160469 learned_addr learned my addr 172.21.15.50:0/3005160469 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:28.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff5186640 1 -- 172.21.15.50:0/3005160469 >> v1:172.21.15.183:6789/0 conn(0x7feff00b38f0 legacy=0x7feff0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:28.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff5186640 1 -- 172.21.15.50:0/3005160469 >> v1:172.21.15.50:6789/0 conn(0x7feff0161210 legacy=0x7feff00b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff5186640 1 -- 172.21.15.50:0/3005160469 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00af7f0 con 0x7feff00b3090 2023-10-19T10:37:28.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.507+0000 7feff4985640 1 -- 172.21.15.50:0/3005160469 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682886814 0 0) 0x7feff00af250 con 0x7feff0161210 2023-10-19T10:37:28.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.508+0000 7feff5186640 1 --2- 172.21.15.50:0/3005160469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00d30e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097a0 tx=0x7fefe002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1df47a1ec63a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.508+0000 7feff4985640 1 -- 172.21.15.50:0/3005160469 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0030610 con 0x7feff00b3090 2023-10-19T10:37:28.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.508+0000 7feff4985640 1 -- 172.21.15.50:0/3005160469 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0030770 con 0x7feff00b3090 2023-10-19T10:37:28.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.508+0000 7feff4985640 1 -- 172.21.15.50:0/3005160469 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0030b00 con 0x7feff00b3090 2023-10-19T10:37:28.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.508+0000 7feff7beb640 1 -- 172.21.15.50:0/3005160469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 msgr2=0x7feff00d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.508+0000 7feff7beb640 1 --2- 172.21.15.50:0/3005160469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00d30e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097a0 tx=0x7fefe002b960 comp rx=0 tx=0).stop 2023-10-19T10:37:28.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.509+0000 7feff7beb640 1 -- 172.21.15.50:0/3005160469 shutdown_connections 2023-10-19T10:37:28.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.509+0000 7feff7beb640 1 --2- 172.21.15.50:0/3005160469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00d30e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.509+0000 7feff7beb640 1 -- 172.21.15.50:0/3005160469 >> 172.21.15.50:0/3005160469 conn(0x7feff009b960 msgr2=0x7feff00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.509+0000 7feff7beb640 1 -- 172.21.15.50:0/3005160469 shutdown_connections 2023-10-19T10:37:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.509+0000 7feff7beb640 1 -- 172.21.15.50:0/3005160469 wait complete. 2023-10-19T10:37:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.510+0000 7feff7beb640 1 Processor -- start 2023-10-19T10:37:28.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.510+0000 7feff7beb640 1 -- start start 2023-10-19T10:37:28.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.510+0000 7feff7beb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.510+0000 7feff7beb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feff00b38f0 0x7feff0157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.510+0000 7feff7beb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feff0161210 0x7feff00cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff7beb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0158470 con 0x7feff00b3090 2023-10-19T10:37:28.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff7beb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00cc540 con 0x7feff0161210 2023-10-19T10:37:28.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff7beb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00caf60 con 0x7feff00b38f0 2023-10-19T10:37:28.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38464/0 (socket says 172.21.15.50:38464) 2023-10-19T10:37:28.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 -- 172.21.15.50:0/2346002052 learned_addr learned my addr 172.21.15.50:0/2346002052 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:28.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5186640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feff00b38f0 0x7feff0157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff6188640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feff0161210 0x7feff00cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 -- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feff00b38f0 msgr2=0x7feff0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feff00b38f0 0x7feff0157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 -- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feff0161210 msgr2=0x7feff00cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feff0161210 0x7feff00cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 -- 172.21.15.50:0/2346002052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0009400 con 0x7feff00b3090 2023-10-19T10:37:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5186640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feff00b38f0 0x7feff0157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff6188640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feff0161210 0x7feff00cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff5987640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00cdec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c450 tx=0x7fefec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0052d0 con 0x7feff00b3090 2023-10-19T10:37:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec005450 con 0x7feff00b3090 2023-10-19T10:37:28.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec003880 con 0x7feff00b3090 2023-10-19T10:37:28.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00cb240 con 0x7feff00b3090 2023-10-19T10:37:28.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.511+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff00cb800 con 0x7feff00b3090 2023-10-19T10:37:28.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.512+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff004e420 con 0x7feff00b3090 2023-10-19T10:37:28.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.513+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fefec0039e0 con 0x7feff00b3090 2023-10-19T10:37:28.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.513+0000 7fefe67fc640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fefd406a050 0x7fefd406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.514+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fefec088340 con 0x7feff00b3090 2023-10-19T10:37:28.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.514+0000 7feff5186640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fefd406a050 0x7fefd406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.514+0000 7feff5186640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fefd406a050 0x7fefd406c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fefe0036000 tx=0x7fefe003b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.518+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fefec0537d0 con 0x7feff00b3090 2023-10-19T10:37:28.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.692+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feff00b2df0 con 0x7feff00b3090 2023-10-19T10:37:28.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.692+0000 7fefe67fc640 1 -- 172.21.15.50:0/2346002052 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fefec056df0 con 0x7feff00b3090 2023-10-19T10:37:28.695 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:28.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fefd406a050 msgr2=0x7fefd406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fefd406a050 0x7fefd406c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fefe0036000 tx=0x7fefe003b040 comp rx=0 tx=0).stop 2023-10-19T10:37:28.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 msgr2=0x7feff00cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00cdec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c450 tx=0x7fefec00c910 comp rx=0 tx=0).stop 2023-10-19T10:37:28.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 shutdown_connections 2023-10-19T10:37:28.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fefd406a050 0x7fefd406c510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feff0161210 0x7feff00cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feff00b38f0 0x7feff0157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 --2- 172.21.15.50:0/2346002052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feff00b3090 0x7feff00cdec0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 >> 172.21.15.50:0/2346002052 conn(0x7feff009b960 msgr2=0x7feff006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:28.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.694+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 shutdown_connections 2023-10-19T10:37:28.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.695+0000 7feff7beb640 1 -- 172.21.15.50:0/2346002052 wait complete. 2023-10-19T10:37:28.708 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:28.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.814+0000 7f9393adc640 1 Processor -- start 2023-10-19T10:37:28.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.814+0000 7f9393adc640 1 -- start start 2023-10-19T10:37:28.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.814+0000 7f9393adc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f938c15cdf0 0x7f938c1675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.814+0000 7f9393adc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c0d3480 con 0x7f938c0b4070 2023-10-19T10:37:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.814+0000 7f9393adc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c15d280 con 0x7f938c15c960 2023-10-19T10:37:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.814+0000 7f9393adc640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c15d560 con 0x7f938c15cdf0 2023-10-19T10:37:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9392079640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f938c15cdf0 0x7f938c1675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9391878640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f938c0b4070 0x7f938c0b4450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41294/0 (socket says 172.21.15.50:41294) 2023-10-19T10:37:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9391878640 1 -- 172.21.15.50:0/1682951387 learned_addr learned my addr 172.21.15.50:0/1682951387 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:28.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9392079640 1 -- 172.21.15.50:0/1682951387 >> v1:172.21.15.183:6789/0 conn(0x7f938c15c960 legacy=0x7f938c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9392079640 1 -- 172.21.15.50:0/1682951387 >> v1:172.21.15.50:6789/0 conn(0x7f938c0b4070 legacy=0x7f938c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9392079640 1 -- 172.21.15.50:0/1682951387 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c15d840 con 0x7f938c15cdf0 2023-10-19T10:37:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.815+0000 7f9390876640 1 -- 172.21.15.50:0/1682951387 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345289286 0 0) 0x7f938c0d3480 con 0x7f938c0b4070 2023-10-19T10:37:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9392079640 1 --2- 172.21.15.50:0/1682951387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c15cdf0 0x7f938c1675b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d90 tx=0x7f938402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48332fe9108638a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9390876640 1 -- 172.21.15.50:0/1682951387 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938402cd00 con 0x7f938c15cdf0 2023-10-19T10:37:28.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9390876640 1 -- 172.21.15.50:0/1682951387 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938402ce60 con 0x7f938c15cdf0 2023-10-19T10:37:28.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9390876640 1 -- 172.21.15.50:0/1682951387 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384034660 con 0x7f938c15cdf0 2023-10-19T10:37:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9393adc640 1 -- 172.21.15.50:0/1682951387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c15cdf0 msgr2=0x7f938c1675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9393adc640 1 --2- 172.21.15.50:0/1682951387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c15cdf0 0x7f938c1675b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d90 tx=0x7f938402bf50 comp rx=0 tx=0).stop 2023-10-19T10:37:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9393adc640 1 -- 172.21.15.50:0/1682951387 shutdown_connections 2023-10-19T10:37:28.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9393adc640 1 --2- 172.21.15.50:0/1682951387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c15cdf0 0x7f938c1675b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9393adc640 1 -- 172.21.15.50:0/1682951387 >> 172.21.15.50:0/1682951387 conn(0x7f938c09b970 msgr2=0x7f938c0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.816+0000 7f9393adc640 1 -- 172.21.15.50:0/1682951387 shutdown_connections 2023-10-19T10:37:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.817+0000 7f9393adc640 1 -- 172.21.15.50:0/1682951387 wait complete. 2023-10-19T10:37:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 Processor -- start 2023-10-19T10:37:28.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 -- start start 2023-10-19T10:37:28.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938c0b4070 0x7f938c1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938c15c960 0x7f938c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 0x7f938c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0c6a30 con 0x7f938c0c65a0 2023-10-19T10:37:28.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0cb400 con 0x7f938c15c960 2023-10-19T10:37:28.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9393adc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0cb6e0 con 0x7f938c0b4070 2023-10-19T10:37:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9392079640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 0x7f938c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9392079640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 0x7f938c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38470/0 (socket says 172.21.15.50:38470) 2023-10-19T10:37:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.818+0000 7f9392079640 1 -- 172.21.15.50:0/3652986759 learned_addr learned my addr 172.21.15.50:0/3652986759 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9391077640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938c15c960 0x7f938c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9391878640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938c0b4070 0x7f938c1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9392079640 1 -- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938c0b4070 msgr2=0x7f938c1f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9392079640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938c0b4070 0x7f938c1f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9392079640 1 -- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938c15c960 msgr2=0x7f938c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9392079640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938c15c960 0x7f938c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9392079640 1 -- 172.21.15.50:0/3652986759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384009a70 con 0x7f938c0c65a0 2023-10-19T10:37:28.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9391878640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938c0b4070 0x7f938c1f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:28.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9392079640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 0x7f938c0ca7b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d90 tx=0x7f9384004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384003b90 con 0x7f938c0c65a0 2023-10-19T10:37:28.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384003cf0 con 0x7f938c0c65a0 2023-10-19T10:37:28.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c0cb9c0 con 0x7f938c0c65a0 2023-10-19T10:37:28.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384004230 con 0x7f938c0c65a0 2023-10-19T10:37:28.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.819+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c0cbf80 con 0x7f938c0c65a0 2023-10-19T10:37:28.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.820+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c0d09b0 con 0x7f938c0c65a0 2023-10-19T10:37:28.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.821+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f938403b050 con 0x7f938c0c65a0 2023-10-19T10:37:28.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.821+0000 7f93827fc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937006a050 0x7f937006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:28.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.821+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f93840ab120 con 0x7f938c0c65a0 2023-10-19T10:37:28.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.821+0000 7f9391878640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937006a050 0x7f937006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:28.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.822+0000 7f9391878640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937006a050 0x7f937006c510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9388001730 tx=0x7f9388009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:28.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:28.825+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9384076520 con 0x7f938c0c65a0 2023-10-19T10:37:29.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.002+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f938c04e420 con 0x7f938c0c65a0 2023-10-19T10:37:29.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.003+0000 7f93827fc640 1 -- 172.21.15.50:0/3652986759 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9384079b40 con 0x7f938c0c65a0 2023-10-19T10:37:29.006 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937006a050 msgr2=0x7f937006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937006a050 0x7f937006c510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9388001730 tx=0x7f9388009040 comp rx=0 tx=0).stop 2023-10-19T10:37:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 msgr2=0x7f938c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 0x7f938c0ca7b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d90 tx=0x7f9384004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 shutdown_connections 2023-10-19T10:37:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f937006a050 0x7f937006c510 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f938c0c65a0 0x7f938c0ca7b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938c15c960 0x7f938c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 --2- 172.21.15.50:0/3652986759 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938c0b4070 0x7f938c1f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 >> 172.21.15.50:0/3652986759 conn(0x7f938c09b970 msgr2=0x7f938c06bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 shutdown_connections 2023-10-19T10:37:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.005+0000 7f9393adc640 1 -- 172.21.15.50:0/3652986759 wait complete. 2023-10-19T10:37:29.018 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:29.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.124+0000 7fb70cacb640 1 Processor -- start 2023-10-19T10:37:29.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.124+0000 7fb70cacb640 1 -- start start 2023-10-19T10:37:29.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70cacb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb7080b38f0 0x7fb7080d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70cacb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b3090 0x7fb708160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70cacb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080af250 con 0x7fb7080b3cd0 2023-10-19T10:37:29.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70cacb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080b0190 con 0x7fb7080b38f0 2023-10-19T10:37:29.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70cacb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080af510 con 0x7fb7080b3090 2023-10-19T10:37:29.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb7080b38f0 0x7fb7080d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb7080b38f0 0x7fb7080d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38486/0 (socket says 172.21.15.50:38486) 2023-10-19T10:37:29.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.125+0000 7fb70659c640 1 -- 172.21.15.50:0/2812101109 learned_addr learned my addr 172.21.15.50:0/2812101109 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:29.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.126+0000 7fb705d9b640 1 --2- 172.21.15.50:0/2812101109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b3090 0x7fb708160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.126+0000 7fb70659c640 1 -- 172.21.15.50:0/2812101109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b3090 msgr2=0x7fb708160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:29.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.126+0000 7fb70659c640 1 --2- 172.21.15.50:0/2812101109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b3090 0x7fb708160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.126+0000 7fb70659c640 1 -- 172.21.15.50:0/2812101109 >> v1:172.21.15.183:6789/0 conn(0x7fb7080b3cd0 legacy=0x7fb7080b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.126+0000 7fb70659c640 1 -- 172.21.15.50:0/2812101109 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7080af7f0 con 0x7fb7080b38f0 2023-10-19T10:37:29.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.126+0000 7fb70659c640 1 --2- 172.21.15.50:0/2812101109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b38f0 0x7fb7080d30e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009ea0 tx=0x7fb6f802bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22789da1afc2121 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70559a640 1 -- 172.21.15.50:0/2812101109 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f802cb40 con 0x7fb7080b38f0 2023-10-19T10:37:29.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70559a640 1 -- 172.21.15.50:0/2812101109 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f802cca0 con 0x7fb7080b38f0 2023-10-19T10:37:29.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70559a640 1 -- 172.21.15.50:0/2812101109 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8034610 con 0x7fb7080b38f0 2023-10-19T10:37:29.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 -- 172.21.15.50:0/2812101109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b38f0 msgr2=0x7fb7080d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2812101109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b38f0 0x7fb7080d30e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009ea0 tx=0x7fb6f802bfa0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 -- 172.21.15.50:0/2812101109 shutdown_connections 2023-10-19T10:37:29.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2812101109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b3090 0x7fb708160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2812101109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b38f0 0x7fb7080d30e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 -- 172.21.15.50:0/2812101109 >> 172.21.15.50:0/2812101109 conn(0x7fb70809b960 msgr2=0x7fb7080d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 -- 172.21.15.50:0/2812101109 shutdown_connections 2023-10-19T10:37:29.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.127+0000 7fb70cacb640 1 -- 172.21.15.50:0/2812101109 wait complete. 2023-10-19T10:37:29.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.128+0000 7fb70cacb640 1 Processor -- start 2023-10-19T10:37:29.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.128+0000 7fb70cacb640 1 -- start start 2023-10-19T10:37:29.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70cacb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7080b3090 0x7fb7080cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70cacb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b38f0 0x7fb7080b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70cacb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 0x7fb708157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70cacb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080ca5f0 con 0x7fb7080b3cd0 2023-10-19T10:37:29.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70cacb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080ca8d0 con 0x7fb7080b3090 2023-10-19T10:37:29.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70cacb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080cabb0 con 0x7fb7080b38f0 2023-10-19T10:37:29.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 0x7fb708157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 0x7fb708157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38502/0 (socket says 172.21.15.50:38502) 2023-10-19T10:37:29.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb706d9d640 1 -- 172.21.15.50:0/2671768774 learned_addr learned my addr 172.21.15.50:0/2671768774 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:29.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb70659c640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7080b3090 0x7fb7080cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.129+0000 7fb705d9b640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b38f0 0x7fb7080b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb706d9d640 1 -- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b38f0 msgr2=0x7fb7080b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb706d9d640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b38f0 0x7fb7080b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb706d9d640 1 -- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7080b3090 msgr2=0x7fb7080cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb706d9d640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7080b3090 0x7fb7080cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb706d9d640 1 -- 172.21.15.50:0/2671768774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f8009b30 con 0x7fb7080b3cd0 2023-10-19T10:37:29.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb70659c640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7080b3090 0x7fb7080cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:29.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb706d9d640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 0x7fb708157850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c390 tx=0x7fb70000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7000052d0 con 0x7fb7080b3cd0 2023-10-19T10:37:29.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700005430 con 0x7fb7080b3cd0 2023-10-19T10:37:29.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700003810 con 0x7fb7080b3cd0 2023-10-19T10:37:29.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7080cae90 con 0x7fb7080b3cd0 2023-10-19T10:37:29.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.130+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb708157d90 con 0x7fb7080b3cd0 2023-10-19T10:37:29.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.131+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7080cb3b0 con 0x7fb7080b3cd0 2023-10-19T10:37:29.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.131+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb700003970 con 0x7fb7080b3cd0 2023-10-19T10:37:29.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.132+0000 7fb6feffd640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb6e806a050 0x7fb6e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.132+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb700087d80 con 0x7fb7080b3cd0 2023-10-19T10:37:29.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.132+0000 7fb70659c640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb6e806a050 0x7fb6e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.132+0000 7fb70659c640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb6e806a050 0x7fb6e806c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8035000 tx=0x7fb6f8030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.136+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb7000531a0 con 0x7fb7080b3cd0 2023-10-19T10:37:29.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.311+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb70804e420 con 0x7fb7080b3cd0 2023-10-19T10:37:29.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.311+0000 7fb6feffd640 1 -- 172.21.15.50:0/2671768774 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb700017020 con 0x7fb7080b3cd0 2023-10-19T10:37:29.314 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:29.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb6e806a050 msgr2=0x7fb6e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb6e806a050 0x7fb6e806c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8035000 tx=0x7fb6f8030000 comp rx=0 tx=0).stop 2023-10-19T10:37:29.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 msgr2=0x7fb708157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 0x7fb708157850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c390 tx=0x7fb70000c850 comp rx=0 tx=0).stop 2023-10-19T10:37:29.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 shutdown_connections 2023-10-19T10:37:29.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb6e806a050 0x7fb6e806c510 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7080b3cd0 0x7fb708157850 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb7080b38f0 0x7fb7080b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 --2- 172.21.15.50:0/2671768774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7080b3090 0x7fb7080cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 >> 172.21.15.50:0/2671768774 conn(0x7fb70809b960 msgr2=0x7fb708158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 shutdown_connections 2023-10-19T10:37:29.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.313+0000 7fb70cacb640 1 -- 172.21.15.50:0/2671768774 wait complete. 2023-10-19T10:37:29.327 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:29.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.433+0000 7ff09ca36640 1 Processor -- start 2023-10-19T10:37:29.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.433+0000 7ff09ca36640 1 -- start start 2023-10-19T10:37:29.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09ca36640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0980b25b0 0x7ff0980ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09ca36640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff09815b500 0x7ff0980cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09ca36640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09815b990 con 0x7ff0980b21d0 2023-10-19T10:37:29.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09ca36640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0980cfc90 con 0x7ff09815b500 2023-10-19T10:37:29.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09ca36640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09815efb0 con 0x7ff0980b25b0 2023-10-19T10:37:29.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09659c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff0980b21d0 0x7ff09806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41310/0 (socket says 172.21.15.50:41310) 2023-10-19T10:37:29.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff09659c640 1 -- 172.21.15.50:0/4239880638 learned_addr learned my addr 172.21.15.50:0/4239880638 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:29.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff096d9d640 1 --2- 172.21.15.50:0/4239880638 >> v2:172.21.15.183:3300/0 conn(0x7ff09815b500 0x7ff0980cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.434+0000 7ff095d9b640 1 --2- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0980b25b0 0x7ff0980ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.435+0000 7ff09559a640 1 -- 172.21.15.50:0/4239880638 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490188071 0 0) 0x7ff09815b990 con 0x7ff0980b21d0 2023-10-19T10:37:29.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.435+0000 7ff09559a640 1 -- 172.21.15.50:0/4239880638 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff080003520 con 0x7ff0980b21d0 2023-10-19T10:37:29.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.435+0000 7ff096d9d640 1 -- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0980b25b0 msgr2=0x7ff0980ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:29.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.435+0000 7ff096d9d640 1 --2- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0980b25b0 0x7ff0980ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.435+0000 7ff096d9d640 1 -- 172.21.15.50:0/4239880638 >> v1:172.21.15.50:6789/0 conn(0x7ff0980b21d0 legacy=0x7ff09806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.435+0000 7ff096d9d640 1 -- 172.21.15.50:0/4239880638 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09815e3a0 con 0x7ff09815b500 2023-10-19T10:37:29.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff096d9d640 1 --2- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff09815b500 0x7ff0980cf750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff090009d90 tx=0x7ff09002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54add1b5ec901e11 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09559a640 1 -- 172.21.15.50:0/4239880638 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09002cc50 con 0x7ff09815b500 2023-10-19T10:37:29.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09559a640 1 -- 172.21.15.50:0/4239880638 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09002cdb0 con 0x7ff09815b500 2023-10-19T10:37:29.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09559a640 1 -- 172.21.15.50:0/4239880638 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090034660 con 0x7ff09815b500 2023-10-19T10:37:29.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 -- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff09815b500 msgr2=0x7ff0980cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 --2- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff09815b500 0x7ff0980cf750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff090009d90 tx=0x7ff09002bea0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 -- 172.21.15.50:0/4239880638 shutdown_connections 2023-10-19T10:37:29.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 --2- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff09815b500 0x7ff0980cf750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 --2- 172.21.15.50:0/4239880638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0980b25b0 0x7ff0980ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 -- 172.21.15.50:0/4239880638 >> 172.21.15.50:0/4239880638 conn(0x7ff0980ccf60 msgr2=0x7ff0980cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 -- 172.21.15.50:0/4239880638 shutdown_connections 2023-10-19T10:37:29.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.436+0000 7ff09ca36640 1 -- 172.21.15.50:0/4239880638 wait complete. 2023-10-19T10:37:29.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 Processor -- start 2023-10-19T10:37:29.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 -- start start 2023-10-19T10:37:29.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 0x7ff0980b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0980b25b0 0x7ff0980ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff09815b500 0x7ff09806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980aec40 con 0x7ff0980b21d0 2023-10-19T10:37:29.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980af3b0 con 0x7ff0980b25b0 2023-10-19T10:37:29.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09ca36640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980b5b90 con 0x7ff09815b500 2023-10-19T10:37:29.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 0x7ff0980b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 0x7ff0980b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38514/0 (socket says 172.21.15.50:38514) 2023-10-19T10:37:29.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.438+0000 7ff09659c640 1 -- 172.21.15.50:0/596264037 learned_addr learned my addr 172.21.15.50:0/596264037 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:29.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff095d9b640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0980b25b0 0x7ff0980ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff096d9d640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff09815b500 0x7ff09806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09659c640 1 -- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff09815b500 msgr2=0x7ff09806f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09659c640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff09815b500 0x7ff09806f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09659c640 1 -- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0980b25b0 msgr2=0x7ff0980ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09659c640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0980b25b0 0x7ff0980ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09659c640 1 -- 172.21.15.50:0/596264037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff090009a70 con 0x7ff0980b21d0 2023-10-19T10:37:29.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09659c640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 0x7ff0980b0140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff08800c500 tx=0x7ff08800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088005ce0 con 0x7ff0980b21d0 2023-10-19T10:37:29.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088005e40 con 0x7ff0980b21d0 2023-10-19T10:37:29.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0980b5e70 con 0x7ff0980b21d0 2023-10-19T10:37:29.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088007330 con 0x7ff0980b21d0 2023-10-19T10:37:29.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.439+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0980aee90 con 0x7ff0980b21d0 2023-10-19T10:37:29.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.440+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09804e420 con 0x7ff0980b21d0 2023-10-19T10:37:29.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.441+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff088007550 con 0x7ff0980b21d0 2023-10-19T10:37:29.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.441+0000 7ff086ffd640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff07806a050 0x7ff07806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.441+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff088087e60 con 0x7ff0980b21d0 2023-10-19T10:37:29.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.441+0000 7ff095d9b640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff07806a050 0x7ff07806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.442+0000 7ff095d9b640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff07806a050 0x7ff07806c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b00b0 tx=0x7ff08c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.445+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff0880533d0 con 0x7ff0980b21d0 2023-10-19T10:37:29.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.619+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff0980cfe50 con 0x7ff0980b21d0 2023-10-19T10:37:29.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.619+0000 7ff086ffd640 1 -- 172.21.15.50:0/596264037 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff0880170a0 con 0x7ff0980b21d0 2023-10-19T10:37:29.623 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:29.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff07806a050 msgr2=0x7ff07806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff07806a050 0x7ff07806c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b00b0 tx=0x7ff08c009190 comp rx=0 tx=0).stop 2023-10-19T10:37:29.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 msgr2=0x7ff0980b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 0x7ff0980b0140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff08800c500 tx=0x7ff08800c9c0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 shutdown_connections 2023-10-19T10:37:29.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff07806a050 0x7ff07806c510 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff09815b500 0x7ff09806f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0980b25b0 0x7ff0980ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 --2- 172.21.15.50:0/596264037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0980b21d0 0x7ff0980b0140 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 >> 172.21.15.50:0/596264037 conn(0x7ff0980ccf60 msgr2=0x7ff0980cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 shutdown_connections 2023-10-19T10:37:29.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.621+0000 7ff09ca36640 1 -- 172.21.15.50:0/596264037 wait complete. 2023-10-19T10:37:29.634 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:37:29.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.738+0000 7fd97e3f7640 1 Processor -- start 2023-10-19T10:37:29.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.738+0000 7fd97e3f7640 1 -- start start 2023-10-19T10:37:29.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd97e3f7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd9780d2db0 0x7fd97815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd97e3f7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97806aa30 con 0x7fd97815c570 2023-10-19T10:37:29.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd97e3f7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97806b2d0 con 0x7fd9780d0e30 2023-10-19T10:37:29.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd97e3f7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97815ce90 con 0x7fd9780d2db0 2023-10-19T10:37:29.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd97c994640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd9780d2db0 0x7fd97815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd977fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd97815c570 0x7fd97806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41314/0 (socket says 172.21.15.50:41314) 2023-10-19T10:37:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd977fff640 1 -- 172.21.15.50:0/83549280 learned_addr learned my addr 172.21.15.50:0/83549280 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.739+0000 7fd97c994640 1 -- 172.21.15.50:0/83549280 >> v1:172.21.15.183:6789/0 conn(0x7fd9780d0e30 legacy=0x7fd9780d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd97c994640 1 -- 172.21.15.50:0/83549280 >> v1:172.21.15.50:6789/0 conn(0x7fd97815c570 legacy=0x7fd97806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd97c994640 1 -- 172.21.15.50:0/83549280 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780d1270 con 0x7fd9780d2db0 2023-10-19T10:37:29.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd976ffd640 1 -- 172.21.15.50:0/83549280 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930907979 0 0) 0x7fd97806aa30 con 0x7fd97815c570 2023-10-19T10:37:29.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd97c994640 1 --2- 172.21.15.50:0/83549280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815c950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd970009d90 tx=0x7fd97002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca509aa7ba7cc4a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd976ffd640 1 -- 172.21.15.50:0/83549280 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97002cc60 con 0x7fd9780d2db0 2023-10-19T10:37:29.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd976ffd640 1 -- 172.21.15.50:0/83549280 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97002cdc0 con 0x7fd9780d2db0 2023-10-19T10:37:29.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.740+0000 7fd976ffd640 1 -- 172.21.15.50:0/83549280 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970034660 con 0x7fd9780d2db0 2023-10-19T10:37:29.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 -- 172.21.15.50:0/83549280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 msgr2=0x7fd97815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/83549280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815c950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd970009d90 tx=0x7fd97002beb0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 -- 172.21.15.50:0/83549280 shutdown_connections 2023-10-19T10:37:29.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/83549280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815c950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 -- 172.21.15.50:0/83549280 >> 172.21.15.50:0/83549280 conn(0x7fd978089ff0 msgr2=0x7fd97809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 -- 172.21.15.50:0/83549280 shutdown_connections 2023-10-19T10:37:29.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.741+0000 7fd97e3f7640 1 -- 172.21.15.50:0/83549280 wait complete. 2023-10-19T10:37:29.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.742+0000 7fd97e3f7640 1 Processor -- start 2023-10-19T10:37:29.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.742+0000 7fd97e3f7640 1 -- start start 2023-10-19T10:37:29.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97e3f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9780d0e30 0x7fd9781f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97e3f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97e3f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97815c570 0x7fd9780cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97e3f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97815e640 con 0x7fd9780d2db0 2023-10-19T10:37:29.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97e3f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780cf2f0 con 0x7fd9780d0e30 2023-10-19T10:37:29.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97e3f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9781f5db0 con 0x7fd97815c570 2023-10-19T10:37:29.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38536/0 (socket says 172.21.15.50:38536) 2023-10-19T10:37:29.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd9777fe640 1 -- 172.21.15.50:0/1789101162 learned_addr learned my addr 172.21.15.50:0/1789101162 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:29.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd977fff640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9780d0e30 0x7fd9781f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.743+0000 7fd97c994640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97815c570 0x7fd9780cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd9777fe640 1 -- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97815c570 msgr2=0x7fd9780cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd9777fe640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97815c570 0x7fd9780cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd9777fe640 1 -- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9780d0e30 msgr2=0x7fd9781f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd9777fe640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9780d0e30 0x7fd9781f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd9777fe640 1 -- 172.21.15.50:0/1789101162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd970009a70 con 0x7fd9780d2db0 2023-10-19T10:37:29.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd97c994640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97815c570 0x7fd9780cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:29.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd9777fe640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815b6b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c400 tx=0x7fd96800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9680052d0 con 0x7fd9780d2db0 2023-10-19T10:37:29.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968005430 con 0x7fd9780d2db0 2023-10-19T10:37:29.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9781f6090 con 0x7fd9780d2db0 2023-10-19T10:37:29.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968003830 con 0x7fd9780d2db0 2023-10-19T10:37:29.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.744+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97814c530 con 0x7fd9780d2db0 2023-10-19T10:37:29.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.745+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97815e4a0 con 0x7fd9780d2db0 2023-10-19T10:37:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.745+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd968003990 con 0x7fd9780d2db0 2023-10-19T10:37:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.746+0000 7fd974ff9640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd96406a050 0x7fd96406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.746+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd968087d60 con 0x7fd9780d2db0 2023-10-19T10:37:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.746+0000 7fd977fff640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd96406a050 0x7fd96406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:29.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.747+0000 7fd977fff640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd96406a050 0x7fd96406c510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd96c007fd0 tx=0x7fd96c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:29.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.750+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd97815e4a0 con 0x7fd9780d2db0 2023-10-19T10:37:29.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.960+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd97815e4a0 con 0x7fd9780d2db0 2023-10-19T10:37:29.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.961+0000 7fd974ff9640 1 -- 172.21.15.50:0/1789101162 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7fd97815e4a0 con 0x7fd9780d2db0 2023-10-19T10:37:29.964 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:29.964 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:37:29.965 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:37:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd96406a050 msgr2=0x7fd96406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd96406a050 0x7fd96406c510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd96c007fd0 tx=0x7fd96c009040 comp rx=0 tx=0).stop 2023-10-19T10:37:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 msgr2=0x7fd97815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:29.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815b6b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c400 tx=0x7fd96800c8c0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 shutdown_connections 2023-10-19T10:37:29.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd96406a050 0x7fd96406c510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97815c570 0x7fd9780cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9780d2db0 0x7fd97815b6b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 --2- 172.21.15.50:0/1789101162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9780d0e30 0x7fd9781f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:29.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 >> 172.21.15.50:0/1789101162 conn(0x7fd978089ff0 msgr2=0x7fd9780d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:29.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 shutdown_connections 2023-10-19T10:37:29.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:29.963+0000 7fd97e3f7640 1 -- 172.21.15.50:0/1789101162 wait complete. 2023-10-19T10:37:29.976 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:30.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.080+0000 7f186a77f640 1 Processor -- start 2023-10-19T10:37:30.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.080+0000 7f186a77f640 1 -- start start 2023-10-19T10:37:30.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f186a77f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f186415c570 0x7f18640d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f186a77f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f18640cd920 0x7f18640d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f186a77f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186406aa30 con 0x7f186415c950 2023-10-19T10:37:30.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f186a77f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186406b2d0 con 0x7f18640cd920 2023-10-19T10:37:30.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f186a77f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18640ce090 con 0x7f186415c570 2023-10-19T10:37:30.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f18640cd920 0x7f18640d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f18640cd920 0x7f18640d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38544/0 (socket says 172.21.15.50:38544) 2023-10-19T10:37:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 -- 172.21.15.50:0/915562287 learned_addr learned my addr 172.21.15.50:0/915562287 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f18637fe640 1 --1- 172.21.15.50:0/915562287 >> v1:172.21.15.50:6789/0 conn(0x7f186415c950 0x7f186406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41320/0 (socket says 172.21.15.50:41320) 2023-10-19T10:37:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1863fff640 1 --2- 172.21.15.50:0/915562287 >> v2:172.21.15.183:3300/0 conn(0x7f186415c570 0x7f18640d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 -- 172.21.15.50:0/915562287 >> v2:172.21.15.183:3300/0 conn(0x7f186415c570 msgr2=0x7f18640d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 --2- 172.21.15.50:0/915562287 >> v2:172.21.15.183:3300/0 conn(0x7f186415c570 0x7f18640d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 -- 172.21.15.50:0/915562287 >> v1:172.21.15.50:6789/0 conn(0x7f186415c950 legacy=0x7f186406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.081+0000 7f1868d1c640 1 -- 172.21.15.50:0/915562287 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186406ac80 con 0x7f18640cd920 2023-10-19T10:37:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.082+0000 7f1868d1c640 1 --2- 172.21.15.50:0/915562287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640cd920 0x7f18640d26a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f18640d2670 tx=0x7f185c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03ea41400157259 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.082+0000 7f1862ffd640 1 -- 172.21.15.50:0/915562287 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c02ca80 con 0x7f18640cd920 2023-10-19T10:37:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.082+0000 7f1862ffd640 1 -- 172.21.15.50:0/915562287 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c02cbe0 con 0x7f18640cd920 2023-10-19T10:37:30.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.082+0000 7f1862ffd640 1 -- 172.21.15.50:0/915562287 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c0343f0 con 0x7f18640cd920 2023-10-19T10:37:30.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.082+0000 7f186a77f640 1 -- 172.21.15.50:0/915562287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640cd920 msgr2=0x7f18640d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.082+0000 7f186a77f640 1 --2- 172.21.15.50:0/915562287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640cd920 0x7f18640d26a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f18640d2670 tx=0x7f185c02bee0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.083+0000 7f186a77f640 1 -- 172.21.15.50:0/915562287 shutdown_connections 2023-10-19T10:37:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.083+0000 7f186a77f640 1 --2- 172.21.15.50:0/915562287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18640cd920 0x7f18640d26a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.083+0000 7f186a77f640 1 --2- 172.21.15.50:0/915562287 >> v2:172.21.15.183:3300/0 conn(0x7f186415c570 0x7f18640d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.083+0000 7f186a77f640 1 -- 172.21.15.50:0/915562287 >> 172.21.15.50:0/915562287 conn(0x7f1864089ff0 msgr2=0x7f186409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.083+0000 7f186a77f640 1 -- 172.21.15.50:0/915562287 shutdown_connections 2023-10-19T10:37:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.083+0000 7f186a77f640 1 -- 172.21.15.50:0/915562287 wait complete. 2023-10-19T10:37:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.084+0000 7f186a77f640 1 Processor -- start 2023-10-19T10:37:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.084+0000 7f186a77f640 1 -- start start 2023-10-19T10:37:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18640cd920 0x7f186409c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 0x7f186415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f186415c950 0x7f186415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186409c970 con 0x7f186415c570 2023-10-19T10:37:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18640ca6d0 con 0x7f18640cd920 2023-10-19T10:37:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186415b710 con 0x7f186415c950 2023-10-19T10:37:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 0x7f186415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 0x7f186415b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38554/0 (socket says 172.21.15.50:38554) 2023-10-19T10:37:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 -- 172.21.15.50:0/1270108840 learned_addr learned my addr 172.21.15.50:0/1270108840 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f1863fff640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18640cd920 0x7f186409c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f1868d1c640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f186415c950 0x7f186415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 -- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f186415c950 msgr2=0x7f186415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f186415c950 0x7f186415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 -- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18640cd920 msgr2=0x7f186409c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18640cd920 0x7f186409c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 -- 172.21.15.50:0/1270108840 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009b30 con 0x7f186415c570 2023-10-19T10:37:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f18637fe640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 0x7f186415b1d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f185800c4d0 tx=0x7f185800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858007480 con 0x7f186415c570 2023-10-19T10:37:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18580075e0 con 0x7f186415c570 2023-10-19T10:37:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18640caa70 con 0x7f186415c570 2023-10-19T10:37:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580049e0 con 0x7f186415c570 2023-10-19T10:37:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.085+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18640cb030 con 0x7f186415c570 2023-10-19T10:37:30.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.086+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1864073990 con 0x7f186415c570 2023-10-19T10:37:30.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.087+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1858004b40 con 0x7f186415c570 2023-10-19T10:37:30.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.087+0000 7f1860ff9640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f185006a050 0x7f185006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.087+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1858087d20 con 0x7f186415c570 2023-10-19T10:37:30.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.088+0000 7f1863fff640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f185006a050 0x7f185006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.088+0000 7f1863fff640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f185006a050 0x7f185006c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1854009b30 tx=0x7f1854009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.092+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f18580531d0 con 0x7f186415c570 2023-10-19T10:37:30.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.268+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18640d3480 con 0x7f186415c570 2023-10-19T10:37:30.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.269+0000 7f1860ff9640 1 -- 172.21.15.50:0/1270108840 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20084 (secure 0 0 0) 0x7f1858073e90 con 0x7f186415c570 2023-10-19T10:37:30.272 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:30.272 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:36:59.714054+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:30.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f185006a050 msgr2=0x7f185006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f185006a050 0x7f185006c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1854009b30 tx=0x7f1854009110 comp rx=0 tx=0).stop 2023-10-19T10:37:30.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 msgr2=0x7f186415b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 0x7f186415b1d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f185800c4d0 tx=0x7f185800c990 comp rx=0 tx=0).stop 2023-10-19T10:37:30.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 shutdown_connections 2023-10-19T10:37:30.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f185006a050 0x7f185006c510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f186415c950 0x7f186415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f186415c570 0x7f186415b1d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 --2- 172.21.15.50:0/1270108840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18640cd920 0x7f186409c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 >> 172.21.15.50:0/1270108840 conn(0x7f1864089ff0 msgr2=0x7f18640b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 shutdown_connections 2023-10-19T10:37:30.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.271+0000 7f186a77f640 1 -- 172.21.15.50:0/1270108840 wait complete. 2023-10-19T10:37:30.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:37:30.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.388+0000 7f720f8c4640 1 Processor -- start 2023-10-19T10:37:30.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.388+0000 7f720f8c4640 1 -- start start 2023-10-19T10:37:30.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f720f8c4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f72080d2db0 0x7f720815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f720f8c4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720806aa30 con 0x7f72080d0e30 2023-10-19T10:37:30.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f720f8c4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720806b2d0 con 0x7f720815c570 2023-10-19T10:37:30.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f720f8c4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720815ce90 con 0x7f72080d2db0 2023-10-19T10:37:30.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f720ce5f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f72080d0e30 0x7f72080d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41326/0 (socket says 172.21.15.50:41326) 2023-10-19T10:37:30.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f720ce5f640 1 -- 172.21.15.50:0/648939927 learned_addr learned my addr 172.21.15.50:0/648939927 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:30.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.389+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108160718 0 0) 0x7f720806aa30 con 0x7f72080d0e30 2023-10-19T10:37:30.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f0003520 con 0x7f72080d0e30 2023-10-19T10:37:30.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720de61640 1 --2- 172.21.15.50:0/648939927 >> v2:172.21.15.183:3300/0 conn(0x7f72080d2db0 0x7f720815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4055017273 0 0) 0x7f71f0003520 con 0x7f72080d0e30 2023-10-19T10:37:30.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 >> v2:172.21.15.183:3300/0 conn(0x7f72080d2db0 msgr2=0x7f720815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:30.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 --2- 172.21.15.50:0/648939927 >> v2:172.21.15.183:3300/0 conn(0x7f72080d2db0 0x7f720815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 >> v1:172.21.15.183:6789/0 conn(0x7f720815c570 legacy=0x7f720806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080d1270 con 0x7f72080d0e30 2023-10-19T10:37:30.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f71fc001670 con 0x7f72080d0e30 2023-10-19T10:37:30.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 >> v1:172.21.15.50:6789/0 conn(0x7f72080d0e30 legacy=0x7f72080d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71f0003970 0x7f71f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f00047c0 0x7f71f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f0005180 0x7f71f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720815ce90 con 0x7f71f0003970 2023-10-19T10:37:30.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000c120 con 0x7f71f0005180 2023-10-19T10:37:30.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000c2e0 con 0x7f71f00047c0 2023-10-19T10:37:30.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720d660640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71f0003970 0x7f71f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720ce5f640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f00047c0 0x7f71f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720de61640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f0005180 0x7f71f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720d660640 1 -- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f00047c0 msgr2=0x7f71f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:30.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720d660640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f00047c0 0x7f71f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720d660640 1 -- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f0005180 msgr2=0x7f71f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:30.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720d660640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f0005180 0x7f71f000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720d660640 1 -- 172.21.15.50:0/648939927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0003520 con 0x7f71f0003970 2023-10-19T10:37:30.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.390+0000 7f720de61640 1 -- 172.21.15.50:0/648939927 reap_dead start 2023-10-19T10:37:30.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.391+0000 7f720d660640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71f0003970 0x7f71f0003d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7200001770 tx=0x7f71f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b96ba171c7977 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:30.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.391+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000052d0 con 0x7f71f0003970 2023-10-19T10:37:30.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.391+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720002e050 con 0x7f71f0003970 2023-10-19T10:37:30.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.391+0000 7f7207fff640 1 -- 172.21.15.50:0/648939927 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200003640 con 0x7f71f0003970 2023-10-19T10:37:30.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 -- 172.21.15.50:0/648939927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71f0003970 msgr2=0x7f71f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71f0003970 0x7f71f0003d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7200001770 tx=0x7f71f00057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 -- 172.21.15.50:0/648939927 shutdown_connections 2023-10-19T10:37:30.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 --2- 172.21.15.50:0/648939927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71f0003970 0x7f71f0003d90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 -- 172.21.15.50:0/648939927 >> 172.21.15.50:0/648939927 conn(0x7f7208089ff0 msgr2=0x7f720809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:30.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 -- 172.21.15.50:0/648939927 shutdown_connections 2023-10-19T10:37:30.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.392+0000 7f720f8c4640 1 -- 172.21.15.50:0/648939927 wait complete. 2023-10-19T10:37:30.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.393+0000 7f720f8c4640 1 Processor -- start 2023-10-19T10:37:30.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.393+0000 7f720f8c4640 1 -- start start 2023-10-19T10:37:30.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720f8c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f0003970 0x7f72080c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720f8c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72081f3070 0x7f72080c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720f8c4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 0x7f72081f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720f8c4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72081f34f0 con 0x7f72081f5710 2023-10-19T10:37:30.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720f8c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080c9390 con 0x7f72081f3070 2023-10-19T10:37:30.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720f8c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72080c90b0 con 0x7f71f0003970 2023-10-19T10:37:30.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 0x7f72081f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 0x7f72081f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38578/0 (socket says 172.21.15.50:38578) 2023-10-19T10:37:30.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 -- 172.21.15.50:0/102181218 learned_addr learned my addr 172.21.15.50:0/102181218 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:30.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720d660640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f0003970 0x7f72080c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720ce5f640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72081f3070 0x7f72080c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 -- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f0003970 msgr2=0x7f72080c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f0003970 0x7f72080c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 -- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72081f3070 msgr2=0x7f72080c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72081f3070 0x7f72080c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 -- 172.21.15.50:0/102181218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080ca960 con 0x7f72081f5710 2023-10-19T10:37:30.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720ce5f640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72081f3070 0x7f72080c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:30.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.394+0000 7f720de61640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 0x7f72081f48a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72080c61e0 tx=0x7f71f800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:30.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.395+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800e040 con 0x7f72081f5710 2023-10-19T10:37:30.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.395+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f800f040 con 0x7f72081f5710 2023-10-19T10:37:30.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.395+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72080cac40 con 0x7f72081f5710 2023-10-19T10:37:30.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.395+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800fec0 con 0x7f72081f5710 2023-10-19T10:37:30.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.395+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72080c8d10 con 0x7f72081f5710 2023-10-19T10:37:30.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.396+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72080cb160 con 0x7f72081f5710 2023-10-19T10:37:30.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.396+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f71f8015430 con 0x7f72081f5710 2023-10-19T10:37:30.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.397+0000 7f7205ffb640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f71f406a080 0x7f71f406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:30.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.397+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f71f8089630 con 0x7f72081f5710 2023-10-19T10:37:30.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.397+0000 7f720d660640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f71f406a080 0x7f71f406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:30.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.397+0000 7f720d660640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f71f406a080 0x7f71f406c540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7200033000 tx=0x7f7200009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:30.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.401+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f72080cb160 con 0x7f72081f5710 2023-10-19T10:37:30.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.612+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72080cb160 con 0x7f72081f5710 2023-10-19T10:37:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.613+0000 7f7205ffb640 1 -- 172.21.15.50:0/102181218 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7f72080cb160 con 0x7f72081f5710 2023-10-19T10:37:30.616 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:30.617 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:37:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:37:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.614+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f71f406a080 msgr2=0x7f71f406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.614+0000 7f720f8c4640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f71f406a080 0x7f71f406c540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7200033000 tx=0x7f7200009030 comp rx=0 tx=0).stop 2023-10-19T10:37:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 msgr2=0x7f72081f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 0x7f72081f48a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72080c61e0 tx=0x7f71f800f1b0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 shutdown_connections 2023-10-19T10:37:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f71f406a080 0x7f71f406c540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72081f5710 0x7f72081f48a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72081f3070 0x7f72080c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 --2- 172.21.15.50:0/102181218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71f0003970 0x7f72080c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 >> 172.21.15.50:0/102181218 conn(0x7f7208089ff0 msgr2=0x7f72080b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 shutdown_connections 2023-10-19T10:37:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:30.615+0000 7f720f8c4640 1 -- 172.21.15.50:0/102181218 wait complete. 2023-10-19T10:37:31.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-19T10:36:54.662444+0000', 'modified': '2023-10-19T10:37:18.122269+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6837/4204689263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6835', 'nonce': 4204689263}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 4204689263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-19T10:37:31.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-19T10:37:31.032 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-19T10:37:31.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.141+0000 7f6e743a0640 1 Processor -- start 2023-10-19T10:37:31.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.141+0000 7f6e743a0640 1 -- start start 2023-10-19T10:37:31.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e743a0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6e6c0cda20 0x7f6e6c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e743a0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6e6c0b1860 0x7f6e6c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e743a0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c160670 con 0x7f6e6c15c6e0 2023-10-19T10:37:31.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e743a0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c161280 con 0x7f6e6c0b1860 2023-10-19T10:37:31.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e743a0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c160950 con 0x7f6e6c0cda20 2023-10-19T10:37:31.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6e6c0b1860 0x7f6e6c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6e6c0b1860 0x7f6e6c0d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38592/0 (socket says 172.21.15.50:38592) 2023-10-19T10:37:31.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 -- 172.21.15.50:0/4097965274 learned_addr learned my addr 172.21.15.50:0/4097965274 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7193b640 1 --2- 172.21.15.50:0/4097965274 >> v2:172.21.15.183:3300/0 conn(0x7f6e6c0cda20 0x7f6e6c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 -- 172.21.15.50:0/4097965274 >> v2:172.21.15.183:3300/0 conn(0x7f6e6c0cda20 msgr2=0x7f6e6c0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 --2- 172.21.15.50:0/4097965274 >> v2:172.21.15.183:3300/0 conn(0x7f6e6c0cda20 0x7f6e6c0d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 -- 172.21.15.50:0/4097965274 >> v1:172.21.15.183:6789/0 conn(0x7f6e6c15c6e0 legacy=0x7f6e6c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7293d640 1 -- 172.21.15.50:0/4097965274 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c160c30 con 0x7f6e6c0b1860 2023-10-19T10:37:31.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.142+0000 7f6e7193b640 1 --2- 172.21.15.50:0/4097965274 >> v2:172.21.15.183:3300/0 conn(0x7f6e6c0cda20 0x7f6e6c0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:31.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.143+0000 7f6e7293d640 1 --2- 172.21.15.50:0/4097965274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0d1200 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c160210 tx=0x7f6e5c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a93eb5c60507a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.143+0000 7f6e7113a640 1 -- 172.21.15.50:0/4097965274 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c02c960 con 0x7f6e6c0b1860 2023-10-19T10:37:31.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.143+0000 7f6e7113a640 1 -- 172.21.15.50:0/4097965274 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c02cac0 con 0x7f6e6c0b1860 2023-10-19T10:37:31.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.143+0000 7f6e7113a640 1 -- 172.21.15.50:0/4097965274 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c02ce50 con 0x7f6e6c0b1860 2023-10-19T10:37:31.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 -- 172.21.15.50:0/4097965274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0b1860 msgr2=0x7f6e6c0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 --2- 172.21.15.50:0/4097965274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0d1200 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c160210 tx=0x7f6e5c02bdc0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 -- 172.21.15.50:0/4097965274 shutdown_connections 2023-10-19T10:37:31.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 --2- 172.21.15.50:0/4097965274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0d1200 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 --2- 172.21.15.50:0/4097965274 >> v2:172.21.15.183:3300/0 conn(0x7f6e6c0cda20 0x7f6e6c0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 -- 172.21.15.50:0/4097965274 >> 172.21.15.50:0/4097965274 conn(0x7f6e6c06a530 msgr2=0x7f6e6c06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 -- 172.21.15.50:0/4097965274 shutdown_connections 2023-10-19T10:37:31.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.144+0000 7f6e743a0640 1 -- 172.21.15.50:0/4097965274 wait complete. 2023-10-19T10:37:31.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.145+0000 7f6e743a0640 1 Processor -- start 2023-10-19T10:37:31.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.145+0000 7f6e743a0640 1 -- start start 2023-10-19T10:37:31.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e743a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e743a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 0x7f6e6c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e743a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e6c15c6e0 0x7f6e6c1f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e743a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0ca350 con 0x7f6e6c0cda20 2023-10-19T10:37:31.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e743a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1514d0 con 0x7f6e6c0b1860 2023-10-19T10:37:31.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e743a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c151770 con 0x7f6e6c15c6e0 2023-10-19T10:37:31.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 0x7f6e6c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 0x7f6e6c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38610/0 (socket says 172.21.15.50:38610) 2023-10-19T10:37:31.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 -- 172.21.15.50:0/2539198267 learned_addr learned my addr 172.21.15.50:0/2539198267 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7213c640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7293d640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e6c15c6e0 0x7f6e6c1f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 -- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e6c15c6e0 msgr2=0x7f6e6c1f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e6c15c6e0 0x7f6e6c1f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 -- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e6c0b1860 msgr2=0x7f6e6c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.146+0000 7f6e7193b640 1 -- 172.21.15.50:0/2539198267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c009b30 con 0x7f6e6c0cda20 2023-10-19T10:37:31.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.147+0000 7f6e7193b640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 0x7f6e6c0c9e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000c7f0 tx=0x7f6e6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.147+0000 7f6e6affd640 1 -- 172.21.15.50:0/2539198267 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60007820 con 0x7f6e6c0cda20 2023-10-19T10:37:31.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.147+0000 7f6e6affd640 1 -- 172.21.15.50:0/2539198267 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60007980 con 0x7f6e6c0cda20 2023-10-19T10:37:31.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.147+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c151a50 con 0x7f6e6c0cda20 2023-10-19T10:37:31.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.147+0000 7f6e6affd640 1 -- 172.21.15.50:0/2539198267 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60007cf0 con 0x7f6e6c0cda20 2023-10-19T10:37:31.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.147+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c0ca580 con 0x7f6e6c0cda20 2023-10-19T10:37:31.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.148+0000 7f6e6affd640 1 -- 172.21.15.50:0/2539198267 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6e600113f0 con 0x7f6e6c0cda20 2023-10-19T10:37:31.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.149+0000 7f6e6affd640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6e5806a050 0x7f6e5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.149+0000 7f6e6affd640 1 -- 172.21.15.50:0/2539198267 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6e60088af0 con 0x7f6e6c0cda20 2023-10-19T10:37:31.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.149+0000 7f6e7213c640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6e5806a050 0x7f6e5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.149+0000 7f6e7213c640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6e5806a050 0x7f6e5806c510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e64004ea0 tx=0x7f6e64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 Processor -- start 2023-10-19T10:37:31.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 -- start start 2023-10-19T10:37:31.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770adcc0 0x5643770ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770b1d50 0x5643770b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564377020060 con 0x5643770dda90 2023-10-19T10:37:31.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437701d750 con 0x5643770b1d50 2023-10-19T10:37:31.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.150+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x564377019040 con 0x5643770adcc0 2023-10-19T10:37:31.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770ddf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38616/0 (socket says 172.21.15.50:38616) 2023-10-19T10:37:31.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 -- 172.21.15.50:0/760326660 learned_addr learned my addr 172.21.15.50:0/760326660 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7213c640 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770adcc0 0x5643770ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7193b640 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770b1d50 0x5643770b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 -- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770adcc0 msgr2=0x5643770ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770adcc0 0x5643770ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 -- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770b1d50 msgr2=0x5643770b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770b1d50 0x5643770b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 -- 172.21.15.50:0/760326660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437701dcb0 con 0x5643770dda90 2023-10-19T10:37:31.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7193b640 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770b1d50 0x5643770b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:31.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e7293d640 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770ddf10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c036040 tx=0x7f6e5c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c98a6bc81735908 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e53fff640 1 -- 172.21.15.50:0/760326660 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c03e630 con 0x5643770dda90 2023-10-19T10:37:31.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e53fff640 1 -- 172.21.15.50:0/760326660 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c03e790 con 0x5643770dda90 2023-10-19T10:37:31.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.151+0000 7f6e53fff640 1 -- 172.21.15.50:0/760326660 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c03eb00 con 0x5643770dda90 2023-10-19T10:37:31.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 -- 172.21.15.50:0/760326660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 msgr2=0x5643770ddf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770ddf10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c036040 tx=0x7f6e5c02e000 comp rx=0 tx=0).stop 2023-10-19T10:37:31.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 -- 172.21.15.50:0/760326660 shutdown_connections 2023-10-19T10:37:31.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770ddf10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770b1d50 0x5643770b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 --2- 172.21.15.50:0/760326660 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770adcc0 0x5643770ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 -- 172.21.15.50:0/760326660 >> 172.21.15.50:0/760326660 conn(0x5643770afd80 msgr2=0x5643770b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 -- 172.21.15.50:0/760326660 shutdown_connections 2023-10-19T10:37:31.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.152+0000 7f6e76feb740 1 -- 172.21.15.50:0/760326660 wait complete. 2023-10-19T10:37:31.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.153+0000 7f6e76feb740 1 -- start start 2023-10-19T10:37:31.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.153+0000 7f6e76feb740 1 Processor -- start 2023-10-19T10:37:31.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.153+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770bd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564376fd5d00 con 0x5643770dda90 2023-10-19T10:37:31.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x564376fd59a0 con 0x5643770dde70 2023-10-19T10:37:31.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x564376f9b1f0 con 0x5643770e10c0 2023-10-19T10:37:31.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 10 client.0 _ll_register_callbacks cb 0x564377010d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-19T10:37:31.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770bd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770bd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38624/0 (socket says 172.21.15.50:38624) 2023-10-19T10:37:31.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 -- 172.21.15.50:0/3814378138 learned_addr learned my addr 172.21.15.50:0/3814378138 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7293d640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7193b640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.175 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 -- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 msgr2=0x5643770e14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770e14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 -- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 msgr2=0x5643770de2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770de2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 -- 172.21.15.50:0/3814378138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c160c30 con 0x5643770dda90 2023-10-19T10:37:31.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e7213c640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770bd940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400ee90 tx=0x7f6e640036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e42590612c82b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e517fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2023-10-19T10:37:31.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e517fa640 1 -- 172.21.15.50:0/3814378138 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64021420 con 0x5643770dda90 2023-10-19T10:37:31.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e517fa640 1 -- 172.21.15.50:0/3814378138 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6e64021580 con 0x5643770dda90 2023-10-19T10:37:31.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564376fede00 con 0x5643770dda90 2023-10-19T10:37:31.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e517fa640 1 -- 172.21.15.50:0/3814378138 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e64003800 con 0x5643770dda90 2023-10-19T10:37:31.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.154+0000 7f6e76feb740 10 client.4923 fetch_fsmap learned FSMap version 9 2023-10-19T10:37:31.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5643770176c0 con 0x5643770dda90 2023-10-19T10:37:31.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e517fa640 1 -- 172.21.15.50:0/3814378138 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564376fede00 con 0x5643770dda90 2023-10-19T10:37:31.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e517fa640 1 -- 172.21.15.50:0/3814378138 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f6e64003960 con 0x5643770dda90 2023-10-19T10:37:31.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 10 client.4923 fetch_fsmap finished waiting for FSMap version 9 2023-10-19T10:37:31.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 10 client.4923 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-19T10:37:31.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 20 client.4923 populate_metadata read hostname 'smithi050' 2023-10-19T10:37:31.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bde80 0x5643770bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 4 client.4923 mds_command: new command op to 4285 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-19T10:37:31.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 --> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564377012020 con 0x5643770bde80 2023-10-19T10:37:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e7193b640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bde80 0x5643770bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e517fa640 10 client.4923 ms_handle_connect on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.155+0000 7f6e7193b640 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bde80 0x5643770bfe50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b969e7d0ccbe4a server_cookie=84aa0bea5d428343 in_seq=0 out_seq=0 2023-10-19T10:37:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e517fa640 1 -- 172.21.15.50:0/3814378138 <== mds.0 v2:172.21.15.183:6835/4204689263 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564377012020 con 0x5643770bde80 2023-10-19T10:37:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e517fa640 10 client.4923 handle_command_reply: tid=0 2023-10-19T10:37:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 client.4923 shutdown 2023-10-19T10:37:31.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 msgr2=0x5643770bd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770bd940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400ee90 tx=0x7f6e640036f0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 shutdown_connections 2023-10-19T10:37:31.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bde80 0x5643770bfe50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770de2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e517fa640 0 client.4923 ms_handle_reset on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 --2- 172.21.15.50:0/3814378138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770bd940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 >> 172.21.15.50:0/3814378138 conn(0x5643770adcc0 msgr2=0x5643770ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 shutdown_connections 2023-10-19T10:37:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.156+0000 7f6e76feb740 1 -- 172.21.15.50:0/3814378138 wait complete. 2023-10-19T10:37:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.157+0000 7f6e76feb740 20 client.4923 trim_cache size 0 max 16384 2023-10-19T10:37:31.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.172+0000 7f6e76feb740 1 Processor -- start 2023-10-19T10:37:31.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.172+0000 7f6e76feb740 1 -- start start 2023-10-19T10:37:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e76feb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643770fa3d0 con 0x5643770dda90 2023-10-19T10:37:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643770fa580 con 0x5643770dde70 2023-10-19T10:37:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643770a9350 con 0x5643770e10c0 2023-10-19T10:37:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38632/0 (socket says 172.21.15.50:38632) 2023-10-19T10:37:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 -- 172.21.15.50:0/4063765639 learned_addr learned my addr 172.21.15.50:0/4063765639 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7193b640 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7293d640 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 -- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 msgr2=0x5643770a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 -- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 msgr2=0x5643770bd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770bd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 -- 172.21.15.50:0/4063765639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564376fb7c20 con 0x5643770dda90 2023-10-19T10:37:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7293d640 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e7213c640 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770b41d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009010 tx=0x7f6e6401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69e53a0ae9205fe server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e68e1c640 1 -- 172.21.15.50:0/4063765639 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64007d70 con 0x5643770dda90 2023-10-19T10:37:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e68e1c640 1 -- 172.21.15.50:0/4063765639 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64004650 con 0x5643770dda90 2023-10-19T10:37:31.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.173+0000 7f6e68e1c640 1 -- 172.21.15.50:0/4063765639 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003800 con 0x5643770dda90 2023-10-19T10:37:31.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 -- 172.21.15.50:0/4063765639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 msgr2=0x5643770b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770b41d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009010 tx=0x7f6e6401f000 comp rx=0 tx=0).stop 2023-10-19T10:37:31.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 -- 172.21.15.50:0/4063765639 shutdown_connections 2023-10-19T10:37:31.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770e10c0 0x5643770a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 --2- 172.21.15.50:0/4063765639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770dda90 0x5643770b41d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 -- 172.21.15.50:0/4063765639 >> 172.21.15.50:0/4063765639 conn(0x5643770adcc0 msgr2=0x5643770de8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 -- 172.21.15.50:0/4063765639 shutdown_connections 2023-10-19T10:37:31.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 -- 172.21.15.50:0/4063765639 wait complete. 2023-10-19T10:37:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 -- start start 2023-10-19T10:37:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.174+0000 7f6e76feb740 1 Processor -- start 2023-10-19T10:37:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770dda90 0x5643770f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 0x5643770d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643770be960 con 0x5643770e10c0 2023-10-19T10:37:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643770d2680 con 0x5643770dde70 2023-10-19T10:37:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643770f92a0 con 0x5643770dda90 2023-10-19T10:37:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e76feb740 10 client.0 _ll_register_callbacks cb 0x7f6e5c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-19T10:37:31.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e7293d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 0x5643770d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.175+0000 7f6e7293d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 0x5643770d7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38638/0 (socket says 172.21.15.50:38638) 2023-10-19T10:37:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 -- 172.21.15.50:0/1166209599 learned_addr learned my addr 172.21.15.50:0/1166209599 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7213c640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770dda90 0x5643770f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7193b640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 -- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770dda90 msgr2=0x5643770f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770dda90 0x5643770f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 -- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 msgr2=0x5643770d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 -- 172.21.15.50:0/1166209599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643770176c0 con 0x5643770e10c0 2023-10-19T10:37:31.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e7293d640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 0x5643770d7f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009f40 tx=0x7f6e5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbc38623d4378a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e517fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2023-10-19T10:37:31.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e517fa640 1 -- 172.21.15.50:0/1166209599 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c03eb00 con 0x5643770e10c0 2023-10-19T10:37:31.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e517fa640 1 -- 172.21.15.50:0/1166209599 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6e5c03ec60 con 0x5643770e10c0 2023-10-19T10:37:31.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e5c03e790 con 0x5643770e10c0 2023-10-19T10:37:31.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e517fa640 1 -- 172.21.15.50:0/1166209599 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e5c03ee20 con 0x5643770e10c0 2023-10-19T10:37:31.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e76feb740 10 client.4929 fetch_fsmap learned FSMap version 9 2023-10-19T10:37:31.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x564376fdd1f0 con 0x5643770e10c0 2023-10-19T10:37:31.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e517fa640 1 -- 172.21.15.50:0/1166209599 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c03e790 con 0x5643770e10c0 2023-10-19T10:37:31.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.176+0000 7f6e517fa640 1 -- 172.21.15.50:0/1166209599 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f6e5c04f070 con 0x5643770e10c0 2023-10-19T10:37:31.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 10 client.4929 fetch_fsmap finished waiting for FSMap version 9 2023-10-19T10:37:31.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 10 client.4929 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-19T10:37:31.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 20 client.4929 populate_metadata read hostname 'smithi050' 2023-10-19T10:37:31.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bf030 0x5643770bf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 4 client.4929 mds_command: new command op to 4285 tid=0[{"prefix": "status"}] 2023-10-19T10:37:31.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 --> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564377011d40 con 0x5643770bf030 2023-10-19T10:37:31.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e7213c640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bf030 0x5643770bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e7213c640 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bf030 0x5643770bf410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5741fc037d0a039d server_cookie=2f2c1585b1c23cc6 in_seq=0 out_seq=0 2023-10-19T10:37:31.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e517fa640 10 client.4929 ms_handle_connect on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:31.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e517fa640 1 -- 172.21.15.50:0/1166209599 <== mds.0 v2:172.21.15.183:6835/4204689263 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564377011d40 con 0x5643770bf030 2023-10-19T10:37:31.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e517fa640 10 client.4929 handle_command_reply: tid=0 2023-10-19T10:37:31.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 1 client.4929 shutdown 2023-10-19T10:37:31.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 msgr2=0x5643770d7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.177+0000 7f6e76feb740 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 0x5643770d7f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009f40 tx=0x7f6e5c004ab0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 shutdown_connections 2023-10-19T10:37:31.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:6835/4204689263,v1:172.21.15.183:6837/4204689263] conn(0x5643770bf030 0x5643770bf410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5643770e10c0 0x5643770d7f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643770dde70 0x5643770d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e517fa640 0 client.4929 ms_handle_reset on v2:172.21.15.183:6835/4204689263 2023-10-19T10:37:31.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 --2- 172.21.15.50:0/1166209599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5643770dda90 0x5643770f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 >> 172.21.15.50:0/1166209599 conn(0x5643770adcc0 msgr2=0x5643770ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 shutdown_connections 2023-10-19T10:37:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 1 -- 172.21.15.50:0/1166209599 wait complete. 2023-10-19T10:37:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e76feb740 20 client.4929 trim_cache size 0 max 16384 2023-10-19T10:37:31.206 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:37:31.206 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "2b47c9ed-f967-49ff-9d46-d95ee8c39a83", 2023-10-19T10:37:31.206 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2023-10-19T10:37:31.206 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4285, 2023-10-19T10:37:31.207 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2023-10-19T10:37:31.207 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2023-10-19T10:37:31.207 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2023-10-19T10:37:31.207 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 35.877225160000002, 2023-10-19T10:37:31.207 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 9, 2023-10-19T10:37:31.208 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 27, 2023-10-19T10:37:31.208 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 27, 2023-10-19T10:37:31.208 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 40.906509217 2023-10-19T10:37:31.208 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:37:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6e5806a050 msgr2=0x7f6e5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6e5806a050 0x7f6e5806c510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e64004ea0 tx=0x7f6e64009040 comp rx=0 tx=0).stop 2023-10-19T10:37:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 msgr2=0x7f6e6c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 0x7f6e6c0c9e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000c7f0 tx=0x7f6e6000ccb0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 shutdown_connections 2023-10-19T10:37:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6e5806a050 0x7f6e5806c510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6e6c15c6e0 0x7f6e6c1f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e6c0cda20 0x7f6e6c0c9e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 --2- 172.21.15.50:0/2539198267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e6c0b1860 0x7f6e6c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 >> 172.21.15.50:0/2539198267 conn(0x7f6e6c06a530 msgr2=0x7f6e6c0d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 shutdown_connections 2023-10-19T10:37:31.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.178+0000 7f6e743a0640 1 -- 172.21.15.50:0/2539198267 wait complete. 2023-10-19T10:37:31.212 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-19T10:37:31.240 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-19T10:37:31.241 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-19T10:37:31.241 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-19T10:37:31.241 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-19T10:37:31.242 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:31.242 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-19T10:37:31.242 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-19T10:37:31.242 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:31.243 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-19T10:37:31.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-19T10:37:31.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:31.243 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-19T10:37:31.244 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-19T10:37:31.244 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:31.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.245 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-10-19T10:37:31.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.264 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2023-10-19T10:37:31.336 INFO:teuthology.orchestra.run.smithi183.stderr:Cannot find device "ceph-brx" 2023-10-19T10:37:31.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:37:31.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.339 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:37:31.356 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-19T10:37:31.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.357 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-19T10:37:31.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.428 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2023-10-19T10:37:31.525 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-19T10:37:31.525 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:37:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.630+0000 7fe3be76a640 1 Processor -- start 2023-10-19T10:37:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.630+0000 7fe3be76a640 1 -- start start 2023-10-19T10:37:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3be76a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe3b8156510 0x7fe3b80ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3be76a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80cadc0 0x7fe3b8159210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3be76a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe3b8159750 0x7fe3b806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3be76a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b806f770 con 0x7fe3b8156510 2023-10-19T10:37:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3be76a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80c95d0 con 0x7fe3b8159750 2023-10-19T10:37:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3be76a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80cb240 con 0x7fe3b80cadc0 2023-10-19T10:37:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe3b8156510 0x7fe3b80ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe3b8156510 0x7fe3b80ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38642/0 (socket says 172.21.15.50:38642) 2023-10-19T10:37:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 -- 172.21.15.50:0/2908603099 learned_addr learned my addr 172.21.15.50:0/2908603099 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3bcd07640 1 --2- 172.21.15.50:0/2908603099 >> v2:172.21.15.183:3300/0 conn(0x7fe3b8159750 0x7fe3b806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b77fe640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80cadc0 0x7fe3b8159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 -- 172.21.15.50:0/2908603099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80cadc0 msgr2=0x7fe3b8159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80cadc0 0x7fe3b8159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 -- 172.21.15.50:0/2908603099 >> v2:172.21.15.183:3300/0 conn(0x7fe3b8159750 msgr2=0x7fe3b806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:31.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 --2- 172.21.15.50:0/2908603099 >> v2:172.21.15.183:3300/0 conn(0x7fe3b8159750 0x7fe3b806b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b7fff640 1 -- 172.21.15.50:0/2908603099 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80c8ca0 con 0x7fe3b8156510 2023-10-19T10:37:31.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.631+0000 7fe3b77fe640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80cadc0 0x7fe3b8159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:31.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.632+0000 7fe3b7fff640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b8156510 0x7fe3b80ca880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe3b806c770 tx=0x7fe3a802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e089fa3186488 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.632+0000 7fe3b6ffd640 1 -- 172.21.15.50:0/2908603099 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a802dba0 con 0x7fe3b8156510 2023-10-19T10:37:31.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.632+0000 7fe3b6ffd640 1 -- 172.21.15.50:0/2908603099 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a802dd00 con 0x7fe3b8156510 2023-10-19T10:37:31.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.632+0000 7fe3b6ffd640 1 -- 172.21.15.50:0/2908603099 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8035660 con 0x7fe3b8156510 2023-10-19T10:37:31.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 -- 172.21.15.50:0/2908603099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b8156510 msgr2=0x7fe3b80ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b8156510 0x7fe3b80ca880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe3b806c770 tx=0x7fe3a802cdf0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 -- 172.21.15.50:0/2908603099 shutdown_connections 2023-10-19T10:37:31.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 --2- 172.21.15.50:0/2908603099 >> v2:172.21.15.183:3300/0 conn(0x7fe3b8159750 0x7fe3b806b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80cadc0 0x7fe3b8159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 --2- 172.21.15.50:0/2908603099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b8156510 0x7fe3b80ca880 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 -- 172.21.15.50:0/2908603099 >> 172.21.15.50:0/2908603099 conn(0x7fe3b8069e90 msgr2=0x7fe3b80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 -- 172.21.15.50:0/2908603099 shutdown_connections 2023-10-19T10:37:31.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.633+0000 7fe3be76a640 1 -- 172.21.15.50:0/2908603099 wait complete. 2023-10-19T10:37:31.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.634+0000 7fe3be76a640 1 Processor -- start 2023-10-19T10:37:31.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.634+0000 7fe3be76a640 1 -- start start 2023-10-19T10:37:31.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3be76a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 0x7fe3b814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3be76a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b8156510 0x7fe3b814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3be76a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b8159750 0x7fe3b80ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3be76a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81f32c0 con 0x7fe3b80cadc0 2023-10-19T10:37:31.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3be76a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81f2990 con 0x7fe3b8159750 2023-10-19T10:37:31.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3be76a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81f2c70 con 0x7fe3b8156510 2023-10-19T10:37:31.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 0x7fe3b814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 0x7fe3b814d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38646/0 (socket says 172.21.15.50:38646) 2023-10-19T10:37:31.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 -- 172.21.15.50:0/1530134820 learned_addr learned my addr 172.21.15.50:0/1530134820 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:31.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3bcd07640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b8159750 0x7fe3b80ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 -- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b8156510 msgr2=0x7fe3b814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b8156510 0x7fe3b814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 -- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b8159750 msgr2=0x7fe3b80ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b8159750 0x7fe3b80ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 -- 172.21.15.50:0/1530134820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a800a670 con 0x7fe3b80cadc0 2023-10-19T10:37:31.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.635+0000 7fe3b7fff640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 0x7fe3b814d730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe3a802ce30 tx=0x7fe3a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.636+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a803f600 con 0x7fe3b80cadc0 2023-10-19T10:37:31.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.636+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a803f760 con 0x7fe3b80cadc0 2023-10-19T10:37:31.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.636+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a803fb10 con 0x7fe3b80cadc0 2023-10-19T10:37:31.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.636+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b81f2f50 con 0x7fe3b80cadc0 2023-10-19T10:37:31.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.636+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b81f25f0 con 0x7fe3b80cadc0 2023-10-19T10:37:31.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.637+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b81f2db0 con 0x7fe3b80cadc0 2023-10-19T10:37:31.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.637+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe3a803fc70 con 0x7fe3b80cadc0 2023-10-19T10:37:31.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.637+0000 7fe3b4ff9640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe3a406a050 0x7fe3a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:31.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.638+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe3a80aae10 con 0x7fe3b80cadc0 2023-10-19T10:37:31.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.638+0000 7fe3b77fe640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe3a406a050 0x7fe3a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:31.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.638+0000 7fe3b77fe640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe3a406a050 0x7fe3a406c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80adcb0 tx=0x7fe3ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:31.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.642+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe3b81f2db0 con 0x7fe3b80cadc0 2023-10-19T10:37:31.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.855+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3b81f2db0 con 0x7fe3b80cadc0 2023-10-19T10:37:31.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.855+0000 7fe3b4ff9640 1 -- 172.21.15.50:0/1530134820 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe3b81f2db0 con 0x7fe3b80cadc0 2023-10-19T10:37:31.858 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-19T10:37:31.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe3a406a050 msgr2=0x7fe3a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe3a406a050 0x7fe3a406c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80adcb0 tx=0x7fe3ac008040 comp rx=0 tx=0).stop 2023-10-19T10:37:31.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 msgr2=0x7fe3b814d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:31.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 0x7fe3b814d730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe3a802ce30 tx=0x7fe3a80046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 shutdown_connections 2023-10-19T10:37:31.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe3a406a050 0x7fe3a406c510 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80adcb0 tx=0x7fe3ac008040 comp rx=0 tx=0).stop 2023-10-19T10:37:31.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b8159750 0x7fe3b80ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b8156510 0x7fe3b814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 --2- 172.21.15.50:0/1530134820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b80cadc0 0x7fe3b814d730 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:31.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 >> 172.21.15.50:0/1530134820 conn(0x7fe3b8069e90 msgr2=0x7fe3b815cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:31.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 shutdown_connections 2023-10-19T10:37:31.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:31.857+0000 7fe3be76a640 1 -- 172.21.15.50:0/1530134820 wait complete. 2023-10-19T10:37:31.870 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:37:31.870 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:37:31.871 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-19T10:37:31.871 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-19T10:37:31.871 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:31.872 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:37:31.872 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:37:31.872 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:37:31.872 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:37:31.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.873 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-19T10:37:31.889 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:31.889 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:31.890 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:31.890 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:31.890 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:31.891 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:31.891 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:31.892 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:31.892 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2023-10-19T10:37:31.892 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:31.893 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:31.893 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2023-10-19T10:37:31.893 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:37:31.893 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:31.893 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2023-10-19T10:37:31.894 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-19T10:37:31.894 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42649sec preferred_lft 42649sec 2023-10-19T10:37:31.894 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2023-10-19T10:37:31.894 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:31.894 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:31.895 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:31.895 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2023-10-19T10:37:31.895 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-19T10:37:31.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:31.897 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:31.897 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:31.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2023-10-19T10:37:31.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2023-10-19T10:37:31.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2023-10-19T10:37:31.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-19T10:37:31.899 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:32.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.050 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:37:32.085 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-19T10:37:32.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.087 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-19T10:37:32.141 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-19T10:37:32.142 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:37:32.142 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-19T10:37:32.142 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-19T10:37:32.142 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:37:32.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.143 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:32.143 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:32.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-19T10:37:32.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-19T10:37:32.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-19T10:37:32.145 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:32.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.250 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:37:32.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.266 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2023-10-19T10:37:32.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.321 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:32.321 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:32.321 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-19T10:37:32.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-19T10:37:32.322 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:32.409 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-19T10:37:32.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.410 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:32.411 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:32.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-19T10:37:32.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-19T10:37:32.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-19T10:37:32.412 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-19T10:37:32.412 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-19T10:37:32.412 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:32.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.515 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:32.515 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:32.516 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2023-10-19T10:37:32.516 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2023-10-19T10:37:32.516 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:32.565 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-19T10:37:32.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:32.566 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:32.581 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:37:32.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:32.582 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:32.636 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:37:32.707 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:32.708 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:32.777 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:32.778 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:32.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.779 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:37:32.834 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-19T10:37:32.835 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-19T10:37:32.877 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:37:32.907 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:32.907 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:32.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:2023-10-19T10:37:32.966+0000 7fdf3d3db480 -1 init, newargv = 0x55cce8e71030 newargc=15 2023-10-19T10:37:32.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[98465]: starting ceph client 2023-10-19T10:37:32.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[98465]: starting fuse 2023-10-19T10:37:32.978 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:32.980 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:32.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:32.981 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:37:33.035 INFO:teuthology.orchestra.run.smithi050.stdout:41 2023-10-19T10:37:33.036 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-19T10:37:33.036 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:37:33.037 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-19T10:37:33.037 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-19T10:37:33.037 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-19T10:37:33.037 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-19T10:37:33.038 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:33.038 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-19T10:37:33.038 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-19T10:37:33.039 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-19T10:37:33.039 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-19T10:37:33.040 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:33.040 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-19T10:37:33.040 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-19T10:37:33.041 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-19T10:37:33.041 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-19T10:37:33.041 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:33.042 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-19T10:37:33.042 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:37:33.042 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:37:33.042 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:37:33.043 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-19T10:37:33.043 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-19T10:37:33.043 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-19T10:37:33.044 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:37:33.044 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:33.044 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.0")) 2023-10-19T10:37:33.044 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-19T10:37:33.160 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.0.98465.asok 2023-10-19T10:37:33.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:37:33.170 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.98465.asok 2023-10-19T10:37:33.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.98465.asok status 2023-10-19T10:37:33.311 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:37:33.312 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-19T10:37:33.312 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:37:33.312 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:37:33.312 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "0", 2023-10-19T10:37:33.313 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-19T10:37:33.313 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-19T10:37:33.313 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "98465", 2023-10-19T10:37:33.313 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-19T10:37:33.313 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:33.314 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-19T10:37:33.314 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-19T10:37:33.314 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4941, 2023-10-19T10:37:33.314 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-19T10:37:33.315 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-19T10:37:33.315 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-19T10:37:33.315 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 4941 2023-10-19T10:37:33.315 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:33.315 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:37:33.316 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:37:33.316 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:37:33.316 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 822759351 2023-10-19T10:37:33.316 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:37:33.316 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:33.317 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:37:33.317 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:37:33.317 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:37:33.317 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 822759351 2023-10-19T10:37:33.317 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:33.318 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.4941 192.168.0.1:0/822759351", 2023-10-19T10:37:33.318 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/822759351", 2023-10-19T10:37:33.318 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-19T10:37:33.318 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 9, 2023-10-19T10:37:33.318 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 27, 2023-10-19T10:37:33.319 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:37:33.320 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-19T10:37:33.320 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-19T10:37:33.320 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:37:33.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:37:33.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.426+0000 7fdccdfb2640 1 Processor -- start 2023-10-19T10:37:33.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.426+0000 7fdccdfb2640 1 -- start start 2023-10-19T10:37:33.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdccdfb2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdcc80cd300 0x7fdcc815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:33.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdccdfb2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdcc80b7fc0 0x7fdcc815c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:33.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdccdfb2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc80b6de0 con 0x7fdcc80cd6e0 2023-10-19T10:37:33.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdccdfb2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc80b7960 con 0x7fdcc80cd300 2023-10-19T10:37:33.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdccdfb2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc80b70c0 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdcc80cd300 0x7fdcc815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:33.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdcc80cd300 0x7fdcc815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38654/0 (socket says 172.21.15.50:38654) 2023-10-19T10:37:33.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 -- 172.21.15.50:0/993099466 learned_addr learned my addr 172.21.15.50:0/993099466 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:33.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc7fff640 1 --2- 172.21.15.50:0/993099466 >> v2:172.21.15.183:3300/0 conn(0x7fdcc80b7fc0 0x7fdcc815c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:33.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 -- 172.21.15.50:0/993099466 >> v2:172.21.15.183:3300/0 conn(0x7fdcc80b7fc0 msgr2=0x7fdcc815c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:33.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 --2- 172.21.15.50:0/993099466 >> v2:172.21.15.183:3300/0 conn(0x7fdcc80b7fc0 0x7fdcc815c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 -- 172.21.15.50:0/993099466 >> v1:172.21.15.183:6789/0 conn(0x7fdcc80cd6e0 legacy=0x7fdcc80d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:33.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.427+0000 7fdcc77fe640 1 -- 172.21.15.50:0/993099466 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80b73a0 con 0x7fdcc80cd300 2023-10-19T10:37:33.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.428+0000 7fdcc77fe640 1 --2- 172.21.15.50:0/993099466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80cd300 0x7fdcc815bff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80b4b20 tx=0x7fdcb802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dc4f7f785de484 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:33.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.428+0000 7fdcc67fc640 1 -- 172.21.15.50:0/993099466 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8030860 con 0x7fdcc80cd300 2023-10-19T10:37:33.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.428+0000 7fdcc67fc640 1 -- 172.21.15.50:0/993099466 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb80309c0 con 0x7fdcc80cd300 2023-10-19T10:37:33.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.428+0000 7fdcc67fc640 1 -- 172.21.15.50:0/993099466 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8030d50 con 0x7fdcc80cd300 2023-10-19T10:37:33.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 -- 172.21.15.50:0/993099466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80cd300 msgr2=0x7fdcc815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:33.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/993099466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80cd300 0x7fdcc815bff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80b4b20 tx=0x7fdcb802b9a0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 -- 172.21.15.50:0/993099466 shutdown_connections 2023-10-19T10:37:33.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/993099466 >> v2:172.21.15.183:3300/0 conn(0x7fdcc80b7fc0 0x7fdcc815c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/993099466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80cd300 0x7fdcc815bff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 -- 172.21.15.50:0/993099466 >> 172.21.15.50:0/993099466 conn(0x7fdcc8071320 msgr2=0x7fdcc8071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:33.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 -- 172.21.15.50:0/993099466 shutdown_connections 2023-10-19T10:37:33.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.429+0000 7fdccdfb2640 1 -- 172.21.15.50:0/993099466 wait complete. 2023-10-19T10:37:33.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.430+0000 7fdccdfb2640 1 Processor -- start 2023-10-19T10:37:33.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.430+0000 7fdccdfb2640 1 -- start start 2023-10-19T10:37:33.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdccdfb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 0x7fdcc8157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:33.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdccdfb2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc80cd300 0x7fdcc81f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:33.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdccdfb2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcc80cd6e0 0x7fdcc80d2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:33.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdccdfb2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc815f000 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdccdfb2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc815e3f0 con 0x7fdcc80cd300 2023-10-19T10:37:33.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdccdfb2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc815e6d0 con 0x7fdcc80cd6e0 2023-10-19T10:37:33.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 0x7fdcc8157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:33.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 0x7fdcc8157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38662/0 (socket says 172.21.15.50:38662) 2023-10-19T10:37:33.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdcc77fe640 1 -- 172.21.15.50:0/1003808603 learned_addr learned my addr 172.21.15.50:0/1003808603 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:33.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdcc7fff640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcc80cd6e0 0x7fdcc80d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:33.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.431+0000 7fdcc6ffd640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc80cd300 0x7fdcc81f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:33.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcc77fe640 1 -- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcc80cd6e0 msgr2=0x7fdcc80d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:33.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcc77fe640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcc80cd6e0 0x7fdcc80d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcc77fe640 1 -- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc80cd300 msgr2=0x7fdcc81f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:33.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcc77fe640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc80cd300 0x7fdcc81f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcc77fe640 1 -- 172.21.15.50:0/1003808603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80094d0 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcc77fe640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 0x7fdcc8157590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000c00 tx=0x7fdcb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:33.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8039c90 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8039df0 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc815e9b0 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8043620 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.432+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc81f4ab0 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.433+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc8073990 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.434+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdcb8043780 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.434+0000 7fdcaffff640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdcb406a050 0x7fdcb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:33.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.434+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdcb803e070 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.434+0000 7fdcc6ffd640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdcb406a050 0x7fdcb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:33.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.435+0000 7fdcc6ffd640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdcb406a050 0x7fdcb406c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80c5c90 tx=0x7fdcbc009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:33.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.438+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fdcb8076d90 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.654+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdcc80d0a60 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.654+0000 7fdcaffff640 1 -- 172.21.15.50:0/1003808603 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdcb807a3b0 con 0x7fdcc80b7fc0 2023-10-19T10:37:33.657 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-19T10:37:33.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdcb406a050 msgr2=0x7fdcb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:33.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdcb406a050 0x7fdcb406c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80c5c90 tx=0x7fdcbc009060 comp rx=0 tx=0).stop 2023-10-19T10:37:33.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 msgr2=0x7fdcc8157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 0x7fdcc8157590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000c00 tx=0x7fdcb8004730 comp rx=0 tx=0).stop 2023-10-19T10:37:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 shutdown_connections 2023-10-19T10:37:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdcb406a050 0x7fdcb406c510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdcc80cd6e0 0x7fdcc80d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc80cd300 0x7fdcc81f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 --2- 172.21.15.50:0/1003808603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcc80b7fc0 0x7fdcc8157590 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:33.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 >> 172.21.15.50:0/1003808603 conn(0x7fdcc8071320 msgr2=0x7fdcc80d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:33.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 shutdown_connections 2023-10-19T10:37:33.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:33.656+0000 7fdccdfb2640 1 -- 172.21.15.50:0/1003808603 wait complete. 2023-10-19T10:37:33.669 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:37:33.670 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:37:33.670 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-19T10:37:33.670 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-19T10:37:33.671 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:33.671 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:37:33.671 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:37:33.672 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:37:33.672 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-19T10:37:33.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.673 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-19T10:37:33.689 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:33.689 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:33.689 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:33.689 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:33.690 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:33.690 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:33.690 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:33.690 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:33.691 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2023-10-19T10:37:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:33.691 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:33.691 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2023-10-19T10:37:33.691 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:37:33.692 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:33.692 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2023-10-19T10:37:33.692 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-19T10:37:33.692 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42647sec preferred_lft 42647sec 2023-10-19T10:37:33.693 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2023-10-19T10:37:33.693 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:33.693 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:33.693 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:33.694 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2023-10-19T10:37:33.694 INFO:teuthology.orchestra.run.smithi050.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:37:33.694 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether fa:5b:82:63:89:b9 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:33.694 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-19T10:37:33.695 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:33.695 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a497:7bff:fe82:fcde/64 scope link tentative 2023-10-19T10:37:33.695 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:33.695 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:37:33.695 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether fa:5b:82:63:89:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-19T10:37:33.696 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::f85b:82ff:fe63:89b9/64 scope link 2023-10-19T10:37:33.696 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:33.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.697 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:37:33.758 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-19T10:37:33.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.760 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-19T10:37:33.816 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-19T10:37:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:37:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-19T10:37:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-19T10:37:33.817 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:37:33.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.818 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:33.818 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:33.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-19T10:37:33.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-19T10:37:33.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-19T10:37:33.819 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:33.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.923 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:37:33.938 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-19T10:37:33.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.939 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2023-10-19T10:37:33.993 INFO:teuthology.orchestra.run.smithi050.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-19T10:37:33.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:33.994 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:33.994 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:33.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-19T10:37:33.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-19T10:37:33.995 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:34.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:34.082 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-19T10:37:34.112 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.114 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:37:34.114 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether a6:c5:05:03:14:8a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-19T10:37:34.114 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-19T10:37:34.114 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.115 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a4c5:5ff:fe03:148a/64 scope link tentative 2023-10-19T10:37:34.115 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:34.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-19T10:37:34.187 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:34.188 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:34.188 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:34.188 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.188 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:34.189 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.189 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:37:34.189 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether a6:c5:05:03:14:8a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-19T10:37:34.189 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-19T10:37:34.189 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.190 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a4c5:5ff:fe03:148a/64 scope link tentative 2023-10-19T10:37:34.190 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:34.191 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-19T10:37:34.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:34.192 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:34.192 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:34.192 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-19T10:37:34.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-19T10:37:34.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-19T10:37:34.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-19T10:37:34.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-19T10:37:34.194 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:34.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:34.335 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:37:34.336 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:37:34.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.1 up 2023-10-19T10:37:34.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.1 master ceph-brx 2023-10-19T10:37:34.337 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:37:34.385 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-19T10:37:34.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:34.387 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:34.402 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-19T10:37:34.402 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:34.402 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:34.458 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:37:34.532 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:34.532 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:34.603 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:34.604 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:34.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:34.605 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:37:34.660 INFO:teuthology.orchestra.run.smithi050.stdout:41 2023-10-19T10:37:34.661 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-10-19T10:37:34.661 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-19T10:37:34.703 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:37:34.734 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:34.735 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:34.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi050.stderr:2023-10-19T10:37:34.796+0000 7f87a1b33480 -1 init, newargv = 0x556ba94cffd0 newargc=15 2023-10-19T10:37:34.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi050.stderr:ceph-fuse[98961]: starting ceph client 2023-10-19T10:37:34.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi050.stderr:ceph-fuse[98961]: starting fuse 2023-10-19T10:37:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:34.809 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:34.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:34.810 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:37:34.864 INFO:teuthology.orchestra.run.smithi050.stdout:41 2023-10-19T10:37:34.865 INFO:teuthology.orchestra.run.smithi050.stdout:44 2023-10-19T10:37:34.865 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-10-19T10:37:34.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:37:34.866 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-19T10:37:34.866 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-19T10:37:34.866 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-19T10:37:34.867 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-19T10:37:34.867 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:34.867 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-19T10:37:34.868 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-19T10:37:34.868 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-19T10:37:34.868 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-19T10:37:34.869 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:34.869 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-19T10:37:34.870 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-19T10:37:34.870 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-19T10:37:34.870 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-19T10:37:34.871 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:34.871 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-19T10:37:34.871 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:37:34.872 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:37:34.872 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:37:34.872 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-19T10:37:34.873 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-19T10:37:34.873 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-19T10:37:34.873 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:37:34.874 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:37:34.874 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.2")) 2023-10-19T10:37:34.875 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-19T10:37:34.990 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.2.98961.asok 2023-10-19T10:37:34.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:37:35.001 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.98961.asok 2023-10-19T10:37:35.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:35.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.98961.asok status 2023-10-19T10:37:35.148 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:37:35.148 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-19T10:37:35.148 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:37:35.148 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:37:35.149 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "2", 2023-10-19T10:37:35.149 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-19T10:37:35.149 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-19T10:37:35.149 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "98961", 2023-10-19T10:37:35.149 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-19T10:37:35.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:35.150 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-19T10:37:35.150 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-19T10:37:35.150 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4956, 2023-10-19T10:37:35.150 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-19T10:37:35.150 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-19T10:37:35.151 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-19T10:37:35.151 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 4956 2023-10-19T10:37:35.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:35.151 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:37:35.151 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:37:35.152 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.2:0", 2023-10-19T10:37:35.152 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2358449748 2023-10-19T10:37:35.152 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:37:35.152 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:35.152 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:37:35.153 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:37:35.153 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.2:0", 2023-10-19T10:37:35.153 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2358449748 2023-10-19T10:37:35.153 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:37:35.153 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.4956 192.168.0.2:0/2358449748", 2023-10-19T10:37:35.153 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.2:0/2358449748", 2023-10-19T10:37:35.154 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-19T10:37:35.154 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 9, 2023-10-19T10:37:35.154 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 27, 2023-10-19T10:37:35.154 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:37:35.154 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-19T10:37:35.155 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-19T10:37:35.155 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:37:35.156 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:37:35.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.262+0000 7fa5d8684640 1 Processor -- start 2023-10-19T10:37:35.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.262+0000 7fa5d8684640 1 -- start start 2023-10-19T10:37:35.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d8684640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa5d0159920 0x7fa5d01638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d8684640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d00c95d0 con 0x7fa5d00caf90 2023-10-19T10:37:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d8684640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d00cd310 con 0x7fa5d0156510 2023-10-19T10:37:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d8684640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d006f770 con 0x7fa5d0159920 2023-10-19T10:37:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d6c21640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa5d0159920 0x7fa5d01638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:35.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d5c1f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa5d00caf90 0x7fa5d0159210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:41338/0 (socket says 172.21.15.50:41338) 2023-10-19T10:37:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d5c1f640 1 -- 172.21.15.50:0/4020840189 learned_addr learned my addr 172.21.15.50:0/4020840189 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d6c21640 1 -- 172.21.15.50:0/4020840189 >> v1:172.21.15.183:6789/0 conn(0x7fa5d0156510 legacy=0x7fa5d00ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d6c21640 1 -- 172.21.15.50:0/4020840189 >> v1:172.21.15.50:6789/0 conn(0x7fa5d00caf90 legacy=0x7fa5d0159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d541e640 1 -- 172.21.15.50:0/4020840189 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837381655 0 0) 0x7fa5d00c95d0 con 0x7fa5d00caf90 2023-10-19T10:37:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.263+0000 7fa5d6c21640 1 -- 172.21.15.50:0/4020840189 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d00c8ca0 con 0x7fa5d0159920 2023-10-19T10:37:35.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.264+0000 7fa5d6c21640 1 --2- 172.21.15.50:0/4020840189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d0159920 0x7fa5d01638f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009d90 tx=0x7fa5c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d47a50340293c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:35.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.264+0000 7fa5d541e640 1 -- 172.21.15.50:0/4020840189 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c802cc60 con 0x7fa5d0159920 2023-10-19T10:37:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.264+0000 7fa5d541e640 1 -- 172.21.15.50:0/4020840189 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c802cdc0 con 0x7fa5d0159920 2023-10-19T10:37:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.264+0000 7fa5d541e640 1 -- 172.21.15.50:0/4020840189 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8034660 con 0x7fa5d0159920 2023-10-19T10:37:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 -- 172.21.15.50:0/4020840189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d0159920 msgr2=0x7fa5d01638f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:35.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 --2- 172.21.15.50:0/4020840189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d0159920 0x7fa5d01638f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009d90 tx=0x7fa5c802beb0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 -- 172.21.15.50:0/4020840189 shutdown_connections 2023-10-19T10:37:35.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 --2- 172.21.15.50:0/4020840189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d0159920 0x7fa5d01638f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 -- 172.21.15.50:0/4020840189 >> 172.21.15.50:0/4020840189 conn(0x7fa5d0069e90 msgr2=0x7fa5d00afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:35.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 -- 172.21.15.50:0/4020840189 shutdown_connections 2023-10-19T10:37:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.265+0000 7fa5d8684640 1 -- 172.21.15.50:0/4020840189 wait complete. 2023-10-19T10:37:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.266+0000 7fa5d8684640 1 Processor -- start 2023-10-19T10:37:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.266+0000 7fa5d8684640 1 -- start start 2023-10-19T10:37:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d8684640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 0x7fa5d0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d8684640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5d0156510 0x7fa5d0151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d8684640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5d0159920 0x7fa5d0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d8684640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00c6350 con 0x7fa5d00caf90 2023-10-19T10:37:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d8684640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00c5c50 con 0x7fa5d0159920 2023-10-19T10:37:35.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d8684640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d0150ca0 con 0x7fa5d0156510 2023-10-19T10:37:35.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 0x7fa5d0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:35.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 0x7fa5d0150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38680/0 (socket says 172.21.15.50:38680) 2023-10-19T10:37:35.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 -- 172.21.15.50:0/2657011368 learned_addr learned my addr 172.21.15.50:0/2657011368 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:35.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d5c1f640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5d0156510 0x7fa5d0151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:35.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6c21640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5d0159920 0x7fa5d0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:35.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 -- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5d0156510 msgr2=0x7fa5d0151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5d0156510 0x7fa5d0151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 -- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5d0159920 msgr2=0x7fa5d0151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5d0159920 0x7fa5d0151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 -- 172.21.15.50:0/2657011368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8009a70 con 0x7fa5d00caf90 2023-10-19T10:37:35.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5d6420640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 0x7fa5d0150760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c430 tx=0x7fa5cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:35.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.267+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc011790 con 0x7fa5d00caf90 2023-10-19T10:37:35.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.268+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0118f0 con 0x7fa5d00caf90 2023-10-19T10:37:35.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.268+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d00c65c0 con 0x7fa5d00caf90 2023-10-19T10:37:35.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.268+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc011ce0 con 0x7fa5d00caf90 2023-10-19T10:37:35.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.268+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d00c6b80 con 0x7fa5d00caf90 2023-10-19T10:37:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.269+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d0073990 con 0x7fa5d00caf90 2023-10-19T10:37:35.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.269+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa5cc01c460 con 0x7fa5d00caf90 2023-10-19T10:37:35.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.270+0000 7fa5c6ffd640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa5b406a050 0x7fa5b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:35.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.270+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa5cc0888f0 con 0x7fa5d00caf90 2023-10-19T10:37:35.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.270+0000 7fa5d5c1f640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa5b406a050 0x7fa5b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:35.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.271+0000 7fa5d5c1f640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa5b406a050 0x7fa5b406c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00c58b0 tx=0x7fa5c0007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:35.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.274+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa5cc0576b0 con 0x7fa5d00caf90 2023-10-19T10:37:35.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.488+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5d00cc0e0 con 0x7fa5d00caf90 2023-10-19T10:37:35.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.488+0000 7fa5c6ffd640 1 -- 172.21.15.50:0/2657011368 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa5cc057240 con 0x7fa5d00caf90 2023-10-19T10:37:35.491 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-19T10:37:35.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa5b406a050 msgr2=0x7fa5b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:35.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa5b406a050 0x7fa5b406c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00c58b0 tx=0x7fa5c0007b50 comp rx=0 tx=0).stop 2023-10-19T10:37:35.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 msgr2=0x7fa5d0150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:35.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 0x7fa5d0150760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c430 tx=0x7fa5cc00c8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 shutdown_connections 2023-10-19T10:37:35.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa5b406a050 0x7fa5b406c510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5d0159920 0x7fa5d0151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5d0156510 0x7fa5d0151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 --2- 172.21.15.50:0/2657011368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5d00caf90 0x7fa5d0150760 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:35.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 >> 172.21.15.50:0/2657011368 conn(0x7fa5d0069e90 msgr2=0x7fa5d0070e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:35.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 shutdown_connections 2023-10-19T10:37:35.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:35.490+0000 7fa5d8684640 1 -- 172.21.15.50:0/2657011368 wait complete. 2023-10-19T10:37:35.503 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:37:35.503 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:37:35.504 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-10-19T10:37:35.504 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-19T10:37:35.504 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:35.504 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:37:35.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:37:35.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:37:35.505 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-19T10:37:35.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:35.506 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-10-19T10:37:35.524 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:35.525 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:35.525 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:35.525 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:35.525 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:35.525 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:35.526 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:35.526 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:35.526 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f0 2023-10-19T10:37:35.526 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:35.526 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:35.527 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f1 2023-10-19T10:37:35.527 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:35.527 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:35.527 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:37:35.527 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:35.528 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-19T10:37:35.528 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42653sec preferred_lft 42653sec 2023-10-19T10:37:35.528 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::ec4:7aff:fe88:8085/64 scope link 2023-10-19T10:37:35.528 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:35.529 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-19T10:37:35.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:35.529 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:35.530 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:35.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add name ceph-brx type bridge 2023-10-19T10:37:35.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip addr flush dev ceph-brx 2023-10-19T10:37:35.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set ceph-brx up 2023-10-19T10:37:35.531 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-19T10:37:35.531 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:35.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:35.797 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:37:35.825 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-10-19T10:37:35.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:35.827 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-10-19T10:37:35.898 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-10-19T10:37:35.898 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:37:35.899 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-19T10:37:35.899 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-19T10:37:35.899 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:37:35.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:35.900 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:35.900 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:35.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-19T10:37:35.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-19T10:37:35.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-19T10:37:35.902 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:36.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.158 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-10-19T10:37:36.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.173 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list-id 2023-10-19T10:37:36.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.228 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:36.228 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:36.229 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-19T10:37:36.229 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-19T10:37:36.229 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:36.312 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-19T10:37:36.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.313 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:36.313 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:36.313 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-19T10:37:36.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-19T10:37:36.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-19T10:37:36.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-19T10:37:36.315 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-19T10:37:36.315 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:36.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.446 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:36.446 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:36.447 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-10-19T10:37:36.447 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set dev brx.0 master ceph-brx 2023-10-19T10:37:36.447 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:36.489 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-19T10:37:36.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:36.490 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:36.544 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-19T10:37:36.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:36.545 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:36.599 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2023-10-19T10:37:36.666 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:36.666 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:36.732 INFO:teuthology.orchestra.run.smithi183.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:36.733 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:36.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.734 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2023-10-19T10:37:36.789 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-19T10:37:36.789 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-19T10:37:36.832 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2023-10-19T10:37:36.858 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:36.858 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:36.924 INFO:teuthology.orchestra.run.smithi183.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:36.925 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:36.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:36.926 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2023-10-19T10:37:36.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:2023-10-19T10:37:36.938+0000 7f8cd7f5a480 -1 init, newargv = 0x564f4f66bb60 newargc=15 2023-10-19T10:37:36.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[94499]: starting ceph client 2023-10-19T10:37:36.944 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[94499]: starting fuse 2023-10-19T10:37:36.956 INFO:teuthology.orchestra.run.smithi183.stdout:41 2023-10-19T10:37:36.956 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-19T10:37:36.957 DEBUG:teuthology.orchestra.run.smithi183:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:37:36.957 DEBUG:teuthology.orchestra.run.smithi183:> import glob 2023-10-19T10:37:36.957 DEBUG:teuthology.orchestra.run.smithi183:> import re 2023-10-19T10:37:36.958 DEBUG:teuthology.orchestra.run.smithi183:> import os 2023-10-19T10:37:36.958 DEBUG:teuthology.orchestra.run.smithi183:> import subprocess 2023-10-19T10:37:36.958 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:36.959 DEBUG:teuthology.orchestra.run.smithi183:> def _find_admin_socket(client_name): 2023-10-19T10:37:36.959 DEBUG:teuthology.orchestra.run.smithi183:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-19T10:37:36.959 DEBUG:teuthology.orchestra.run.smithi183:> files = glob.glob(asok_path) 2023-10-19T10:37:36.960 DEBUG:teuthology.orchestra.run.smithi183:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-19T10:37:36.960 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:36.960 DEBUG:teuthology.orchestra.run.smithi183:> # Given a non-glob path, it better be there 2023-10-19T10:37:36.961 DEBUG:teuthology.orchestra.run.smithi183:> if "*" not in asok_path: 2023-10-19T10:37:36.961 DEBUG:teuthology.orchestra.run.smithi183:> assert(len(files) == 1) 2023-10-19T10:37:36.961 DEBUG:teuthology.orchestra.run.smithi183:> return files[0] 2023-10-19T10:37:36.962 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:36.962 DEBUG:teuthology.orchestra.run.smithi183:> for f in files: 2023-10-19T10:37:36.963 DEBUG:teuthology.orchestra.run.smithi183:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:37:36.963 DEBUG:teuthology.orchestra.run.smithi183:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:37:36.964 DEBUG:teuthology.orchestra.run.smithi183:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:37:36.964 DEBUG:teuthology.orchestra.run.smithi183:> contents = proc_f.read() 2023-10-19T10:37:36.964 DEBUG:teuthology.orchestra.run.smithi183:> if mountpoint in contents: 2023-10-19T10:37:36.964 DEBUG:teuthology.orchestra.run.smithi183:> return f 2023-10-19T10:37:36.965 DEBUG:teuthology.orchestra.run.smithi183:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:37:36.965 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:36.965 DEBUG:teuthology.orchestra.run.smithi183:> print(_find_admin_socket("client.1")) 2023-10-19T10:37:36.966 DEBUG:teuthology.orchestra.run.smithi183:> ' 2023-10-19T10:37:37.078 INFO:teuthology.orchestra.run.smithi183.stdout:/var/run/ceph/ceph-client.1.94499.asok 2023-10-19T10:37:37.080 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:37:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:37:37.086 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94499.asok 2023-10-19T10:37:37.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94499.asok status 2023-10-19T10:37:37.227 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-10-19T10:37:37.227 INFO:teuthology.orchestra.run.smithi183.stdout: "metadata": { 2023-10-19T10:37:37.227 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:37:37.228 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:37:37.228 INFO:teuthology.orchestra.run.smithi183.stdout: "entity_id": "1", 2023-10-19T10:37:37.229 INFO:teuthology.orchestra.run.smithi183.stdout: "hostname": "smithi183", 2023-10-19T10:37:37.229 INFO:teuthology.orchestra.run.smithi183.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-19T10:37:37.229 INFO:teuthology.orchestra.run.smithi183.stdout: "pid": "94499", 2023-10-19T10:37:37.229 INFO:teuthology.orchestra.run.smithi183.stdout: "root": "/" 2023-10-19T10:37:37.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:37.230 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_count": 0, 2023-10-19T10:37:37.230 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_pinned_count": 0, 2023-10-19T10:37:37.231 INFO:teuthology.orchestra.run.smithi183.stdout: "id": 4643, 2023-10-19T10:37:37.231 INFO:teuthology.orchestra.run.smithi183.stdout: "inst": { 2023-10-19T10:37:37.231 INFO:teuthology.orchestra.run.smithi183.stdout: "name": { 2023-10-19T10:37:37.232 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "client", 2023-10-19T10:37:37.232 INFO:teuthology.orchestra.run.smithi183.stdout: "num": 4643 2023-10-19T10:37:37.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:37.233 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2023-10-19T10:37:37.233 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2023-10-19T10:37:37.234 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:37:37.234 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 4023645871 2023-10-19T10:37:37.235 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-10-19T10:37:37.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:37.235 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2023-10-19T10:37:37.235 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2023-10-19T10:37:37.236 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:37:37.236 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 4023645871 2023-10-19T10:37:37.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:37.237 INFO:teuthology.orchestra.run.smithi183.stdout: "inst_str": "client.4643 192.168.0.1:0/4023645871", 2023-10-19T10:37:37.237 INFO:teuthology.orchestra.run.smithi183.stdout: "addr_str": "192.168.0.1:0/4023645871", 2023-10-19T10:37:37.237 INFO:teuthology.orchestra.run.smithi183.stdout: "inode_count": 1, 2023-10-19T10:37:37.237 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_epoch": 9, 2023-10-19T10:37:37.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch": 29, 2023-10-19T10:37:37.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:37:37.238 INFO:teuthology.orchestra.run.smithi183.stdout: "blocklisted": false, 2023-10-19T10:37:37.239 INFO:teuthology.orchestra.run.smithi183.stdout: "fs_name": "cephfs" 2023-10-19T10:37:37.239 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-10-19T10:37:37.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:37:37.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.356+0000 7f1cb2b18640 1 Processor -- start 2023-10-19T10:37:37.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.356+0000 7f1cb2b18640 1 -- start start 2023-10-19T10:37:37.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cb2b18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:37.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cb2b18640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac06aa30 con 0x7f1cac0d0e30 2023-10-19T10:37:37.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cb2b18640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac06b2d0 con 0x7f1cac15c570 2023-10-19T10:37:37.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cb2b18640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15ce90 con 0x7f1cac0d2db0 2023-10-19T10:37:37.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cabfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1cac0d0e30 0x7f1cac0d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40606/0 (socket says 172.21.15.50:40606) 2023-10-19T10:37:37.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cabfff640 1 -- 172.21.15.50:0/709316703 learned_addr learned my addr 172.21.15.50:0/709316703 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:37.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.357+0000 7f1cb10b5640 1 --2- 172.21.15.50:0/709316703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:37.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663369292 0 0) 0x7f1cac06aa30 con 0x7f1cac0d0e30 2023-10-19T10:37:37.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c98003520 con 0x7f1cac0d0e30 2023-10-19T10:37:37.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425423886 0 0) 0x7f1cac06b2d0 con 0x7f1cac15c570 2023-10-19T10:37:37.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cac06aa30 con 0x7f1cac15c570 2023-10-19T10:37:37.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cb10b5640 1 -- 172.21.15.50:0/709316703 >> v1:172.21.15.183:6789/0 conn(0x7f1cac15c570 legacy=0x7f1cac06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cb10b5640 1 -- 172.21.15.50:0/709316703 >> v1:172.21.15.50:6789/0 conn(0x7f1cac0d0e30 legacy=0x7f1cac0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654767842 0 0) 0x7f1c98003520 con 0x7f1cac0d0e30 2023-10-19T10:37:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.358+0000 7f1cb10b5640 1 -- 172.21.15.50:0/709316703 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0d1270 con 0x7f1cac0d2db0 2023-10-19T10:37:37.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb10b5640 1 --2- 172.21.15.50:0/709316703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac15c950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c940097d0 tx=0x7f1c9402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99edde9ab7ae133e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:37.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c001670 con 0x7f1cac0d2db0 2023-10-19T10:37:37.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9402e050 con 0x7f1cac0d2db0 2023-10-19T10:37:37.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cab7fe640 1 -- 172.21.15.50:0/709316703 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94030730 con 0x7f1cac0d2db0 2023-10-19T10:37:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 -- 172.21.15.50:0/709316703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 msgr2=0x7f1cac15c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/709316703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac15c950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c940097d0 tx=0x7f1c9402b770 comp rx=0 tx=0).stop 2023-10-19T10:37:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 -- 172.21.15.50:0/709316703 shutdown_connections 2023-10-19T10:37:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/709316703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac15c950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 -- 172.21.15.50:0/709316703 >> 172.21.15.50:0/709316703 conn(0x7f1cac089ff0 msgr2=0x7f1cac09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 -- 172.21.15.50:0/709316703 shutdown_connections 2023-10-19T10:37:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.359+0000 7f1cb2b18640 1 -- 172.21.15.50:0/709316703 wait complete. 2023-10-19T10:37:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.361+0000 7f1cb2b18640 1 Processor -- start 2023-10-19T10:37:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.361+0000 7f1cb2b18640 1 -- start start 2023-10-19T10:37:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0d0e30 0x7f1cac15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 0x7f1cac0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b8100 con 0x7f1cac15c570 2023-10-19T10:37:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b7e60 con 0x7f1cac0d0e30 2023-10-19T10:37:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0cb710 con 0x7f1cac0d2db0 2023-10-19T10:37:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 0x7f1cac0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 0x7f1cac0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56760/0 (socket says 172.21.15.50:56760) 2023-10-19T10:37:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 -- 172.21.15.50:0/960308557 learned_addr learned my addr 172.21.15.50:0/960308557 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb08b4640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0d0e30 0x7f1cac15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cabfff640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 -- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 msgr2=0x7f1cac0b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac0b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 -- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0d0e30 msgr2=0x7f1cac15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0d0e30 0x7f1cac15ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 -- 172.21.15.50:0/960308557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94009400 con 0x7f1cac15c570 2023-10-19T10:37:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb10b5640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 0x7f1cac0cc040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009720 tx=0x7f1c940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:37.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94004230 con 0x7f1cac15c570 2023-10-19T10:37:37.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94004390 con 0x7f1cac15c570 2023-10-19T10:37:37.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac0b5dc0 con 0x7f1cac15c570 2023-10-19T10:37:37.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94003850 con 0x7f1cac15c570 2023-10-19T10:37:37.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.362+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac0b6380 con 0x7f1cac15c570 2023-10-19T10:37:37.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.364+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac073990 con 0x7f1cac15c570 2023-10-19T10:37:37.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.364+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c94039040 con 0x7f1cac15c570 2023-10-19T10:37:37.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.364+0000 7f1ca97fa640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c8c06a080 0x7f1c8c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:37.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.365+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1c940a9c00 con 0x7f1cac15c570 2023-10-19T10:37:37.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.365+0000 7f1cb08b4640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c8c06a080 0x7f1c8c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:37.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.365+0000 7f1cb08b4640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c8c06a080 0x7f1c8c06c540 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40015b0 tx=0x7f1ca4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:37.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.369+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1c940b0080 con 0x7f1cac15c570 2023-10-19T10:37:37.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.580+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1cac0d3480 con 0x7f1cac15c570 2023-10-19T10:37:37.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.580+0000 7f1ca97fa640 1 -- 172.21.15.50:0/960308557 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1c94037070 con 0x7f1cac15c570 2023-10-19T10:37:37.583 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-19T10:37:37.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c8c06a080 msgr2=0x7f1c8c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c8c06a080 0x7f1c8c06c540 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40015b0 tx=0x7f1ca4009040 comp rx=0 tx=0).stop 2023-10-19T10:37:37.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 msgr2=0x7f1cac0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:37.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 0x7f1cac0cc040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009720 tx=0x7f1c940046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 shutdown_connections 2023-10-19T10:37:37.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c8c06a080 0x7f1c8c06c540 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cac15c570 0x7f1cac0cc040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cac0d2db0 0x7f1cac0b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 --2- 172.21.15.50:0/960308557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cac0d0e30 0x7f1cac15ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 >> 172.21.15.50:0/960308557 conn(0x7f1cac089ff0 msgr2=0x7f1cac0d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:37.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 shutdown_connections 2023-10-19T10:37:37.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:37.582+0000 7f1cb2b18640 1 -- 172.21.15.50:0/960308557 wait complete. 2023-10-19T10:37:37.595 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:37:37.596 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:37:37.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-10-19T10:37:37.596 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-19T10:37:37.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:37.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:37:37.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:37:37.597 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:37:37.597 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-19T10:37:37.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.598 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-10-19T10:37:37.615 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:37.615 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:37.615 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:37.615 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:37.616 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:37.616 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:37.616 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:37.616 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:37.616 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f0 2023-10-19T10:37:37.616 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:37.617 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:37.617 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f1 2023-10-19T10:37:37.617 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:37:37.617 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:37.618 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:37:37.618 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:37.618 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-19T10:37:37.618 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42651sec preferred_lft 42651sec 2023-10-19T10:37:37.619 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::ec4:7aff:fe88:8085/64 scope link 2023-10-19T10:37:37.619 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:37.619 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:37:37.620 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether ea:e2:d8:35:f6:0f brd ff:ff:ff:ff:ff:ff 2023-10-19T10:37:37.620 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-19T10:37:37.620 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:37.620 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc40:d8ff:fe0c:e190/64 scope link tentative 2023-10-19T10:37:37.620 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:37.621 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:37:37.621 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether ea:e2:d8:35:f6:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-19T10:37:37.621 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::e8e2:d8ff:fe35:f60f/64 scope link tentative 2023-10-19T10:37:37.621 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:37.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.622 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:37:37.682 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-10-19T10:37:37.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.683 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-10-19T10:37:37.739 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-10-19T10:37:37.739 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:37:37.739 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-19T10:37:37.740 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-19T10:37:37.740 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:37:37.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.740 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:37.741 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:37.741 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-19T10:37:37.741 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-19T10:37:37.742 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-19T10:37:37.742 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:37.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.835 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-10-19T10:37:37.849 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-19T10:37:37.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.851 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list-id 2023-10-19T10:37:37.905 INFO:teuthology.orchestra.run.smithi183.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-19T10:37:37.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.906 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:37.906 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:37.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-19T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-19T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:37.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:37.989 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-19T10:37:38.053 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:38.053 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:38.053 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:38.053 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.054 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:38.054 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.054 INFO:teuthology.orchestra.run.smithi183.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:37:38.054 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether ea:e9:08:78:d1:58 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-19T10:37:38.054 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-19T10:37:38.055 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.055 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::e8e9:8ff:fe78:d158/64 scope link 2023-10-19T10:37:38.055 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:38.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-19T10:37:38.122 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:37:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:37:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:37:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-10-19T10:37:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.123 INFO:teuthology.orchestra.run.smithi183.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:37:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether ea:e9:08:78:d1:58 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-19T10:37:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-19T10:37:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::e8e9:8ff:fe78:d158/64 scope link 2023-10-19T10:37:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:37:38.125 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-19T10:37:38.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:38.126 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:38.126 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:38.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-19T10:37:38.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-19T10:37:38.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-19T10:37:38.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-19T10:37:38.128 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-19T10:37:38.128 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:38.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:38.250 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:37:38.251 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:37:38.251 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.1 up 2023-10-19T10:37:38.251 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set dev brx.1 master ceph-brx 2023-10-19T10:37:38.252 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:37:38.293 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-19T10:37:38.293 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:38.294 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:38.347 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-19T10:37:38.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:37:38.348 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:38.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2023-10-19T10:37:38.469 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:38.470 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:38.535 INFO:teuthology.orchestra.run.smithi183.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:38.536 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:38.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:38.537 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2023-10-19T10:37:38.592 INFO:teuthology.orchestra.run.smithi183.stdout:41 2023-10-19T10:37:38.593 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-10-19T10:37:38.593 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-19T10:37:38.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2023-10-19T10:37:38.661 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:37:38.661 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:37:38.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi183.stderr:2023-10-19T10:37:38.716+0000 7fadc3e6a480 -1 init, newargv = 0x565187c6a040 newargc=15 2023-10-19T10:37:38.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi183.stderr:ceph-fuse[94934]: starting ceph client 2023-10-19T10:37:38.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi183.stderr:ceph-fuse[94934]: starting fuse 2023-10-19T10:37:38.728 INFO:teuthology.orchestra.run.smithi183.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:37:38.729 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:37:38.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:38.730 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2023-10-19T10:37:38.785 INFO:teuthology.orchestra.run.smithi183.stdout:41 2023-10-19T10:37:38.785 INFO:teuthology.orchestra.run.smithi183.stdout:44 2023-10-19T10:37:38.785 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-10-19T10:37:38.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:37:38.786 DEBUG:teuthology.orchestra.run.smithi183:> import glob 2023-10-19T10:37:38.786 DEBUG:teuthology.orchestra.run.smithi183:> import re 2023-10-19T10:37:38.787 DEBUG:teuthology.orchestra.run.smithi183:> import os 2023-10-19T10:37:38.787 DEBUG:teuthology.orchestra.run.smithi183:> import subprocess 2023-10-19T10:37:38.787 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:38.788 DEBUG:teuthology.orchestra.run.smithi183:> def _find_admin_socket(client_name): 2023-10-19T10:37:38.788 DEBUG:teuthology.orchestra.run.smithi183:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-19T10:37:38.788 DEBUG:teuthology.orchestra.run.smithi183:> files = glob.glob(asok_path) 2023-10-19T10:37:38.789 DEBUG:teuthology.orchestra.run.smithi183:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-19T10:37:38.789 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:38.789 DEBUG:teuthology.orchestra.run.smithi183:> # Given a non-glob path, it better be there 2023-10-19T10:37:38.790 DEBUG:teuthology.orchestra.run.smithi183:> if "*" not in asok_path: 2023-10-19T10:37:38.790 DEBUG:teuthology.orchestra.run.smithi183:> assert(len(files) == 1) 2023-10-19T10:37:38.790 DEBUG:teuthology.orchestra.run.smithi183:> return files[0] 2023-10-19T10:37:38.791 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:38.791 DEBUG:teuthology.orchestra.run.smithi183:> for f in files: 2023-10-19T10:37:38.791 DEBUG:teuthology.orchestra.run.smithi183:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:37:38.791 DEBUG:teuthology.orchestra.run.smithi183:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:37:38.792 DEBUG:teuthology.orchestra.run.smithi183:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:37:38.792 DEBUG:teuthology.orchestra.run.smithi183:> contents = proc_f.read() 2023-10-19T10:37:38.792 DEBUG:teuthology.orchestra.run.smithi183:> if mountpoint in contents: 2023-10-19T10:37:38.793 DEBUG:teuthology.orchestra.run.smithi183:> return f 2023-10-19T10:37:38.793 DEBUG:teuthology.orchestra.run.smithi183:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:37:38.793 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:37:38.794 DEBUG:teuthology.orchestra.run.smithi183:> print(_find_admin_socket("client.3")) 2023-10-19T10:37:38.794 DEBUG:teuthology.orchestra.run.smithi183:> ' 2023-10-19T10:37:38.904 INFO:teuthology.orchestra.run.smithi183.stdout:/var/run/ceph/ceph-client.3.94934.asok 2023-10-19T10:37:38.907 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:37:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:37:38.914 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.94934.asok 2023-10-19T10:37:38.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:38.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.94934.asok status 2023-10-19T10:37:39.049 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-10-19T10:37:39.050 INFO:teuthology.orchestra.run.smithi183.stdout: "metadata": { 2023-10-19T10:37:39.050 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:37:39.050 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:37:39.050 INFO:teuthology.orchestra.run.smithi183.stdout: "entity_id": "3", 2023-10-19T10:37:39.051 INFO:teuthology.orchestra.run.smithi183.stdout: "hostname": "smithi183", 2023-10-19T10:37:39.051 INFO:teuthology.orchestra.run.smithi183.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-19T10:37:39.051 INFO:teuthology.orchestra.run.smithi183.stdout: "pid": "94934", 2023-10-19T10:37:39.051 INFO:teuthology.orchestra.run.smithi183.stdout: "root": "/" 2023-10-19T10:37:39.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:39.052 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_count": 0, 2023-10-19T10:37:39.052 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_pinned_count": 0, 2023-10-19T10:37:39.052 INFO:teuthology.orchestra.run.smithi183.stdout: "id": 4652, 2023-10-19T10:37:39.052 INFO:teuthology.orchestra.run.smithi183.stdout: "inst": { 2023-10-19T10:37:39.052 INFO:teuthology.orchestra.run.smithi183.stdout: "name": { 2023-10-19T10:37:39.053 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "client", 2023-10-19T10:37:39.053 INFO:teuthology.orchestra.run.smithi183.stdout: "num": 4652 2023-10-19T10:37:39.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:39.053 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2023-10-19T10:37:39.053 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2023-10-19T10:37:39.053 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.2:0", 2023-10-19T10:37:39.054 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 984382661 2023-10-19T10:37:39.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-10-19T10:37:39.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:39.054 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2023-10-19T10:37:39.054 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2023-10-19T10:37:39.055 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.2:0", 2023-10-19T10:37:39.055 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 984382661 2023-10-19T10:37:39.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:37:39.055 INFO:teuthology.orchestra.run.smithi183.stdout: "inst_str": "client.4652 192.168.0.2:0/984382661", 2023-10-19T10:37:39.055 INFO:teuthology.orchestra.run.smithi183.stdout: "addr_str": "192.168.0.2:0/984382661", 2023-10-19T10:37:39.056 INFO:teuthology.orchestra.run.smithi183.stdout: "inode_count": 1, 2023-10-19T10:37:39.056 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_epoch": 9, 2023-10-19T10:37:39.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch": 29, 2023-10-19T10:37:39.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:37:39.056 INFO:teuthology.orchestra.run.smithi183.stdout: "blocklisted": false, 2023-10-19T10:37:39.057 INFO:teuthology.orchestra.run.smithi183.stdout: "fs_name": "cephfs" 2023-10-19T10:37:39.057 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-10-19T10:37:39.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.058 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-19T10:37:39.059 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:39.078 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-19T10:37:39.079 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:39.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:39.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.157 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-19T10:37:39.157 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:39.215 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-19T10:37:39.215 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:39.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-19T10:37:39.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.290 DEBUG:teuthology.orchestra.run.smithi183:> stat --file-system '--printf=%T 2023-10-19T10:37:39.290 DEBUG:teuthology.orchestra.run.smithi183:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:39.306 INFO:teuthology.orchestra.run.smithi183.stdout:fuseblk 2023-10-19T10:37:39.306 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:39.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.307 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-19T10:37:39.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.375 DEBUG:teuthology.orchestra.run.smithi183:> stat --file-system '--printf=%T 2023-10-19T10:37:39.375 DEBUG:teuthology.orchestra.run.smithi183:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:39.430 INFO:teuthology.orchestra.run.smithi183.stdout:fuseblk 2023-10-19T10:37:39.431 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:39.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:39.431 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-19T10:37:39.497 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-19T10:37:39.539 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.654+0000 7f39b9654640 1 Processor -- start 2023-10-19T10:37:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.654+0000 7f39b9654640 1 -- start start 2023-10-19T10:37:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b9654640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39b40ccc30 0x7f39b40cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b9654640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40b5460 0x7f39b40b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b9654640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b40b19c0 con 0x7f39b415c550 2023-10-19T10:37:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b9654640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40d28e0 con 0x7f39b40ccc30 2023-10-19T10:37:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b9654640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40d1a30 con 0x7f39b40b5460 2023-10-19T10:37:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b27fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39b40ccc30 0x7f39b40cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b2ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f39b415c550 0x7f39b40d0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40612/0 (socket says 172.21.15.50:40612) 2023-10-19T10:37:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.655+0000 7f39b2ffd640 1 -- 172.21.15.50:0/953767733 learned_addr learned my addr 172.21.15.50:0/953767733 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b37fe640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40b5460 0x7f39b40b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b27fc640 1 -- 172.21.15.50:0/953767733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40b5460 msgr2=0x7f39b40b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b27fc640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40b5460 0x7f39b40b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b27fc640 1 -- 172.21.15.50:0/953767733 >> v1:172.21.15.50:6789/0 conn(0x7f39b415c550 legacy=0x7f39b40d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b1ffb640 1 -- 172.21.15.50:0/953767733 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769400363 0 0) 0x7f39b40b19c0 con 0x7f39b415c550 2023-10-19T10:37:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b27fc640 1 -- 172.21.15.50:0/953767733 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40cd5d0 con 0x7f39b40ccc30 2023-10-19T10:37:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.656+0000 7f39b37fe640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40b5460 0x7f39b40b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b27fc640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40ccc30 0x7f39b40cd090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f39a00097a0 tx=0x7f39a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465d8f68b76f3c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b1ffb640 1 -- 172.21.15.50:0/953767733 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0030610 con 0x7f39b40ccc30 2023-10-19T10:37:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b1ffb640 1 -- 172.21.15.50:0/953767733 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a0030770 con 0x7f39b40ccc30 2023-10-19T10:37:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b1ffb640 1 -- 172.21.15.50:0/953767733 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0030b00 con 0x7f39b40ccc30 2023-10-19T10:37:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 -- 172.21.15.50:0/953767733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40ccc30 msgr2=0x7f39b40cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40ccc30 0x7f39b40cd090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f39a00097a0 tx=0x7f39a002b960 comp rx=0 tx=0).stop 2023-10-19T10:37:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 -- 172.21.15.50:0/953767733 shutdown_connections 2023-10-19T10:37:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40b5460 0x7f39b40b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 --2- 172.21.15.50:0/953767733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40ccc30 0x7f39b40cd090 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 -- 172.21.15.50:0/953767733 >> 172.21.15.50:0/953767733 conn(0x7f39b4071320 msgr2=0x7f39b4071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 -- 172.21.15.50:0/953767733 shutdown_connections 2023-10-19T10:37:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.657+0000 7f39b9654640 1 -- 172.21.15.50:0/953767733 wait complete. 2023-10-19T10:37:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.659+0000 7f39b9654640 1 Processor -- start 2023-10-19T10:37:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.659+0000 7f39b9654640 1 -- start start 2023-10-19T10:37:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b9654640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 0x7f39b41f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b9654640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40ccc30 0x7f39b41f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b9654640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39b415c550 0x7f39b40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b9654640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b415c930 con 0x7f39b40b5460 2023-10-19T10:37:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b9654640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b415d1d0 con 0x7f39b415c550 2023-10-19T10:37:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b9654640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41f2ba0 con 0x7f39b40ccc30 2023-10-19T10:37:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 0x7f39b41f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 0x7f39b41f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56780/0 (socket says 172.21.15.50:56780) 2023-10-19T10:37:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 -- 172.21.15.50:0/3610952493 learned_addr learned my addr 172.21.15.50:0/3610952493 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b27fc640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40ccc30 0x7f39b41f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b37fe640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39b415c550 0x7f39b40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 -- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40ccc30 msgr2=0x7f39b41f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40ccc30 0x7f39b41f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 -- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39b415c550 msgr2=0x7f39b40c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39b415c550 0x7f39b40c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b2ffd640 1 -- 172.21.15.50:0/3610952493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a0009400 con 0x7f39b40b5460 2023-10-19T10:37:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.660+0000 7f39b27fc640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40ccc30 0x7f39b41f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.661+0000 7f39b2ffd640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 0x7f39b41f2f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c460 tx=0x7f39a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:39.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.661+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40074e0 con 0x7f39b40b5460 2023-10-19T10:37:39.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.661+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4007660 con 0x7f39b40b5460 2023-10-19T10:37:39.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.661+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b415cb80 con 0x7f39b40b5460 2023-10-19T10:37:39.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.661+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004a20 con 0x7f39b40b5460 2023-10-19T10:37:39.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.661+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b41f1b10 con 0x7f39b40b5460 2023-10-19T10:37:39.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.662+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b40d2d50 con 0x7f39b40b5460 2023-10-19T10:37:39.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.662+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f39a4004b80 con 0x7f39b40b5460 2023-10-19T10:37:39.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.663+0000 7f39ab7fe640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f399c06a050 0x7f399c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.663+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f39a4087ad0 con 0x7f39b40b5460 2023-10-19T10:37:39.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.663+0000 7f39b27fc640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f399c06a050 0x7f399c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.663+0000 7f39b27fc640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f399c06a050 0x7f399c06c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39a0036000 tx=0x7f39a0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:39.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.667+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f39b40d2d50 con 0x7f39b40b5460 2023-10-19T10:37:39.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.844+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39b40d2d50 con 0x7f39b40b5460 2023-10-19T10:37:39.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.845+0000 7f39ab7fe640 1 -- 172.21.15.50:0/3610952493 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f39b40d2d50 con 0x7f39b40b5460 2023-10-19T10:37:39.848 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:39.848 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:39.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f399c06a050 msgr2=0x7f399c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f399c06a050 0x7f399c06c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39a0036000 tx=0x7f39a0008030 comp rx=0 tx=0).stop 2023-10-19T10:37:39.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 msgr2=0x7f39b41f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 0x7f39b41f2f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c460 tx=0x7f39a400c920 comp rx=0 tx=0).stop 2023-10-19T10:37:39.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 shutdown_connections 2023-10-19T10:37:39.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f399c06a050 0x7f399c06c510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39b415c550 0x7f39b40c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39b40ccc30 0x7f39b41f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 --2- 172.21.15.50:0/3610952493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b40b5460 0x7f39b41f2f10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 >> 172.21.15.50:0/3610952493 conn(0x7f39b4071320 msgr2=0x7f39b40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:39.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 shutdown_connections 2023-10-19T10:37:39.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.847+0000 7f39b9654640 1 -- 172.21.15.50:0/3610952493 wait complete. 2023-10-19T10:37:39.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:39.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:39.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 Processor -- start 2023-10-19T10:37:39.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 -- start start 2023-10-19T10:37:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff9140b3090 0x7ff9140d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff914161210 0x7ff9140b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9140af250 con 0x7ff9140b38f0 2023-10-19T10:37:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140b0190 con 0x7ff9140b3090 2023-10-19T10:37:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.968+0000 7ff919251640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140af510 con 0x7ff914161210 2023-10-19T10:37:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff91259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff9140b3090 0x7ff9140d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff912d9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff9140b38f0 0x7ff914160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40618/0 (socket says 172.21.15.50:40618) 2023-10-19T10:37:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff912d9d640 1 -- 172.21.15.50:0/2881400492 learned_addr learned my addr 172.21.15.50:0/2881400492 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff91359e640 1 --2- 172.21.15.50:0/2881400492 >> v2:172.21.15.183:3300/0 conn(0x7ff914161210 0x7ff9140b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff91259c640 1 -- 172.21.15.50:0/2881400492 >> v2:172.21.15.183:3300/0 conn(0x7ff914161210 msgr2=0x7ff9140b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff91259c640 1 --2- 172.21.15.50:0/2881400492 >> v2:172.21.15.183:3300/0 conn(0x7ff914161210 0x7ff9140b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff91259c640 1 -- 172.21.15.50:0/2881400492 >> v1:172.21.15.50:6789/0 conn(0x7ff9140b38f0 legacy=0x7ff914160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff911d9b640 1 -- 172.21.15.50:0/2881400492 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884906182 0 0) 0x7ff9140af250 con 0x7ff9140b38f0 2023-10-19T10:37:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.969+0000 7ff91259c640 1 -- 172.21.15.50:0/2881400492 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140af7f0 con 0x7ff9140b3090 2023-10-19T10:37:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff91259c640 1 --2- 172.21.15.50:0/2881400492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b3090 0x7ff9140d30e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9140b3470 tx=0x7ff90802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742d1d0d2ae9cc2b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff911d9b640 1 -- 172.21.15.50:0/2881400492 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908031830 con 0x7ff9140b3090 2023-10-19T10:37:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff911d9b640 1 -- 172.21.15.50:0/2881400492 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908031990 con 0x7ff9140b3090 2023-10-19T10:37:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff911d9b640 1 -- 172.21.15.50:0/2881400492 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908031d20 con 0x7ff9140b3090 2023-10-19T10:37:39.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 -- 172.21.15.50:0/2881400492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b3090 msgr2=0x7ff9140d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 --2- 172.21.15.50:0/2881400492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b3090 0x7ff9140d30e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9140b3470 tx=0x7ff90802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 -- 172.21.15.50:0/2881400492 shutdown_connections 2023-10-19T10:37:39.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 --2- 172.21.15.50:0/2881400492 >> v2:172.21.15.183:3300/0 conn(0x7ff914161210 0x7ff9140b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 --2- 172.21.15.50:0/2881400492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b3090 0x7ff9140d30e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 -- 172.21.15.50:0/2881400492 >> 172.21.15.50:0/2881400492 conn(0x7ff91409b960 msgr2=0x7ff9140d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:39.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 -- 172.21.15.50:0/2881400492 shutdown_connections 2023-10-19T10:37:39.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.970+0000 7ff919251640 1 -- 172.21.15.50:0/2881400492 wait complete. 2023-10-19T10:37:39.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.972+0000 7ff919251640 1 Processor -- start 2023-10-19T10:37:39.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.972+0000 7ff919251640 1 -- start start 2023-10-19T10:37:39.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff919251640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9140b3090 0x7ff9141f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff919251640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 0x7ff91415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff919251640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914161210 0x7ff9140c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff919251640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141f3300 con 0x7ff9140b38f0 2023-10-19T10:37:39.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff919251640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140ca680 con 0x7ff914161210 2023-10-19T10:37:39.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff919251640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140cc160 con 0x7ff9140b3090 2023-10-19T10:37:39.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 0x7ff91415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 0x7ff91415a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56798/0 (socket says 172.21.15.50:56798) 2023-10-19T10:37:39.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 -- 172.21.15.50:0/1710486687 learned_addr learned my addr 172.21.15.50:0/1710486687 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:39.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91359e640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914161210 0x7ff9140c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff912d9d640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9140b3090 0x7ff9141f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 -- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9140b3090 msgr2=0x7ff9141f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9140b3090 0x7ff9141f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 -- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914161210 msgr2=0x7ff9140c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:39.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914161210 0x7ff9140c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:39.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 -- 172.21.15.50:0/1710486687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908009400 con 0x7ff9140b38f0 2023-10-19T10:37:39.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff912d9d640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9140b3090 0x7ff9141f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:39.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.973+0000 7ff91259c640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 0x7ff91415a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff908009890 tx=0x7ff9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:39.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.974+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908004230 con 0x7ff9140b38f0 2023-10-19T10:37:39.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.974+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9080043b0 con 0x7ff9140b38f0 2023-10-19T10:37:39.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.974+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91415a920 con 0x7ff9140b38f0 2023-10-19T10:37:39.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.974+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908004520 con 0x7ff9140b38f0 2023-10-19T10:37:39.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.974+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9140cb3a0 con 0x7ff9140b38f0 2023-10-19T10:37:39.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.975+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91404e420 con 0x7ff9140b38f0 2023-10-19T10:37:39.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.975+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff90803a8a0 con 0x7ff9140b38f0 2023-10-19T10:37:39.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.976+0000 7ff8fb7fe640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff90006a050 0x7ff90006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:39.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.976+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff9080aaf40 con 0x7ff9140b38f0 2023-10-19T10:37:39.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.976+0000 7ff912d9d640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff90006a050 0x7ff90006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:39.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.976+0000 7ff912d9d640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff90006a050 0x7ff90006c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff9040016f0 tx=0x7ff904009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:39.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:39.980+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff908076300 con 0x7ff9140b38f0 2023-10-19T10:37:40.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.155+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9140b86b0 con 0x7ff9140b38f0 2023-10-19T10:37:40.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.155+0000 7ff8fb7fe640 1 -- 172.21.15.50:0/1710486687 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff908079920 con 0x7ff9140b38f0 2023-10-19T10:37:40.158 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff90006a050 msgr2=0x7ff90006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff90006a050 0x7ff90006c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff9040016f0 tx=0x7ff904009040 comp rx=0 tx=0).stop 2023-10-19T10:37:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 msgr2=0x7ff91415a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 0x7ff91415a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff908009890 tx=0x7ff9080046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 shutdown_connections 2023-10-19T10:37:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff90006a050 0x7ff90006c510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914161210 0x7ff9140c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9140b38f0 0x7ff91415a3e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 --2- 172.21.15.50:0/1710486687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9140b3090 0x7ff9141f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 >> 172.21.15.50:0/1710486687 conn(0x7ff91409b960 msgr2=0x7ff91406bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 shutdown_connections 2023-10-19T10:37:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.157+0000 7ff919251640 1 -- 172.21.15.50:0/1710486687 wait complete. 2023-10-19T10:37:40.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:40.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 Processor -- start 2023-10-19T10:37:40.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 -- start start 2023-10-19T10:37:40.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c0b38f0 0x7f8a0c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8a0c161210 0x7f8a0c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0af250 con 0x7f8a0c0b3090 2023-10-19T10:37:40.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0b0190 con 0x7f8a0c161210 2023-10-19T10:37:40.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.278+0000 7f8a10828640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0af510 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a0ad9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8a0c161210 0x7f8a0c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a03fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f8a0c0b3090 0x7f8a0c160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40622/0 (socket says 172.21.15.50:40622) 2023-10-19T10:37:40.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a03fff640 1 -- 172.21.15.50:0/557433737 learned_addr learned my addr 172.21.15.50:0/557433737 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:40.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a0a59c640 1 --2- 172.21.15.50:0/557433737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c0b38f0 0x7f8a0c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a0ad9d640 1 -- 172.21.15.50:0/557433737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c0b38f0 msgr2=0x7f8a0c0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a0ad9d640 1 --2- 172.21.15.50:0/557433737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c0b38f0 0x7f8a0c0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a0ad9d640 1 -- 172.21.15.50:0/557433737 >> v1:172.21.15.50:6789/0 conn(0x7f8a0c0b3090 legacy=0x7f8a0c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a0ad9d640 1 -- 172.21.15.50:0/557433737 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0af7f0 con 0x7f8a0c161210 2023-10-19T10:37:40.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.279+0000 7f8a09d9b640 1 -- 172.21.15.50:0/557433737 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112134640 0 0) 0x7f8a0c0af250 con 0x7f8a0c0b3090 2023-10-19T10:37:40.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a0ad9d640 1 --2- 172.21.15.50:0/557433737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c161210 0x7f8a0c0b05b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22ec663510dbbc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a09d9b640 1 -- 172.21.15.50:0/557433737 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402cc50 con 0x7f8a0c161210 2023-10-19T10:37:40.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a09d9b640 1 -- 172.21.15.50:0/557433737 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0402cdb0 con 0x7f8a0c161210 2023-10-19T10:37:40.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a09d9b640 1 -- 172.21.15.50:0/557433737 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04034660 con 0x7f8a0c161210 2023-10-19T10:37:40.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 -- 172.21.15.50:0/557433737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c161210 msgr2=0x7f8a0c0b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 --2- 172.21.15.50:0/557433737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c161210 0x7f8a0c0b05b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0402bea0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 -- 172.21.15.50:0/557433737 shutdown_connections 2023-10-19T10:37:40.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 --2- 172.21.15.50:0/557433737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c161210 0x7f8a0c0b05b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 --2- 172.21.15.50:0/557433737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c0b38f0 0x7f8a0c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 -- 172.21.15.50:0/557433737 >> 172.21.15.50:0/557433737 conn(0x7f8a0c09b960 msgr2=0x7f8a0c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:40.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 -- 172.21.15.50:0/557433737 shutdown_connections 2023-10-19T10:37:40.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.280+0000 7f8a10828640 1 -- 172.21.15.50:0/557433737 wait complete. 2023-10-19T10:37:40.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 Processor -- start 2023-10-19T10:37:40.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 -- start start 2023-10-19T10:37:40.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c0b3090 0x7f8a0c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 0x7f8a0c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c161210 0x7f8a0c15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c15d9b0 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c15b870 con 0x7f8a0c0b3090 2023-10-19T10:37:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a10828640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0ca470 con 0x7f8a0c161210 2023-10-19T10:37:40.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 0x7f8a0c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 0x7f8a0c15adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56808/0 (socket says 172.21.15.50:56808) 2023-10-19T10:37:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.282+0000 7f8a03fff640 1 -- 172.21.15.50:0/1964162603 learned_addr learned my addr 172.21.15.50:0/1964162603 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a0ad9d640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c161210 0x7f8a0c15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a0a59c640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c0b3090 0x7f8a0c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a03fff640 1 -- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c161210 msgr2=0x7f8a0c15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a03fff640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c161210 0x7f8a0c15b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a03fff640 1 -- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c0b3090 msgr2=0x7f8a0c157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a03fff640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c0b3090 0x7f8a0c157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a03fff640 1 -- 172.21.15.50:0/1964162603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009a70 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a03fff640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 0x7f8a0c15adf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c4b0 tx=0x7f89f400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40074e0 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4007660 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0ca710 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4004a20 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.283+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c0cacd0 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.284+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c0caef0 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.285+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f89f4004b80 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.285+0000 7f8a02ffd640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89ec06a050 0x7f89ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.285+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f89f4087d20 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.285+0000 7f8a0a59c640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89ec06a050 0x7f89ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.286+0000 7f8a0a59c640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89ec06a050 0x7f89ec06c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b9b0 tx=0x7f89fc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.289+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f89f4053230 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.467+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8a0c0afaa0 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.467+0000 7f8a02ffd640 1 -- 172.21.15.50:0/1964162603 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89f40180a0 con 0x7f8a0c0b38f0 2023-10-19T10:37:40.470 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:40.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89ec06a050 msgr2=0x7f89ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89ec06a050 0x7f89ec06c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b9b0 tx=0x7f89fc008040 comp rx=0 tx=0).stop 2023-10-19T10:37:40.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 msgr2=0x7f8a0c15adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 0x7f8a0c15adf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c4b0 tx=0x7f89f400c970 comp rx=0 tx=0).stop 2023-10-19T10:37:40.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 shutdown_connections 2023-10-19T10:37:40.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89ec06a050 0x7f89ec06c510 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a0c161210 0x7f8a0c15b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0c0b38f0 0x7f8a0c15adf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 --2- 172.21.15.50:0/1964162603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c0b3090 0x7f8a0c157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 >> 172.21.15.50:0/1964162603 conn(0x7f8a0c09b960 msgr2=0x7f8a0c158270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:40.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 shutdown_connections 2023-10-19T10:37:40.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.469+0000 7f8a10828640 1 -- 172.21.15.50:0/1964162603 wait complete. 2023-10-19T10:37:40.482 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:40.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.589+0000 7f2bba9e9640 1 Processor -- start 2023-10-19T10:37:40.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.589+0000 7f2bba9e9640 1 -- start start 2023-10-19T10:37:40.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bba9e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2bb40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bba9e9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2bb415c960 0x7f2bb40d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bba9e9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b4990 con 0x7f2bb415cdc0 2023-10-19T10:37:40.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bba9e9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb415d280 con 0x7f2bb415c960 2023-10-19T10:37:40.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bba9e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb415d560 con 0x7f2bb40b4070 2023-10-19T10:37:40.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb8f86640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2bb415cdc0 0x7f2bb4160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40628/0 (socket says 172.21.15.50:40628) 2023-10-19T10:37:40.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb8f86640 1 -- 172.21.15.50:0/2784179971 learned_addr learned my addr 172.21.15.50:0/2784179971 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:40.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb3fff640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2bb40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb37fe640 1 --2- 172.21.15.50:0/2784179971 >> v2:172.21.15.183:3300/0 conn(0x7f2bb415c960 0x7f2bb40d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144515461 0 0) 0x7f2bb40b4990 con 0x7f2bb415cdc0 2023-10-19T10:37:40.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b9c003520 con 0x7f2bb415cdc0 2023-10-19T10:37:40.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3709767099 0 0) 0x7f2b9c003520 con 0x7f2bb415cdc0 2023-10-19T10:37:40.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 msgr2=0x7f2bb40b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2bb40b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 >> v2:172.21.15.183:3300/0 conn(0x7f2bb415c960 msgr2=0x7f2bb40d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 --2- 172.21.15.50:0/2784179971 >> v2:172.21.15.183:3300/0 conn(0x7f2bb415c960 0x7f2bb40d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb415d840 con 0x7f2bb415cdc0 2023-10-19T10:37:40.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f2bac001a10 con 0x7f2bb415cdc0 2023-10-19T10:37:40.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 >> v1:172.21.15.50:6789/0 conn(0x7f2bb415cdc0 legacy=0x7f2bb4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c003970 0x7f2b9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.590+0000 7f2bb2ffd640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b9c0047c0 0x7f2b9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb2ffd640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2b9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb415d280 con 0x7f2b9c0047c0 2023-10-19T10:37:40.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb415d560 con 0x7f2b9c003970 2023-10-19T10:37:40.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c00b980 con 0x7f2bb40b4070 2023-10-19T10:37:40.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb37fe640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b9c0047c0 0x7f2b9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb3fff640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c003970 0x7f2b9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb8f86640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2b9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb37fe640 1 -- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 msgr2=0x7f2b9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb37fe640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2b9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb37fe640 1 -- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c003970 msgr2=0x7f2b9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb37fe640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c003970 0x7f2b9c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb37fe640 1 -- 172.21.15.50:0/2784179971 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c003520 con 0x7f2b9c0047c0 2023-10-19T10:37:40.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.591+0000 7f2bb8f86640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40b4070 0x7f2b9c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:40.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bb37fe640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b9c0047c0 0x7f2b9c004c40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009ac0 tx=0x7f2b9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d8301a9f1d22be server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba402c710 con 0x7f2b9c0047c0 2023-10-19T10:37:40.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba402c870 con 0x7f2b9c0047c0 2023-10-19T10:37:40.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bb2ffd640 1 -- 172.21.15.50:0/2784179971 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba402cc00 con 0x7f2b9c0047c0 2023-10-19T10:37:40.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bba9e9640 1 -- 172.21.15.50:0/2784179971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b9c0047c0 msgr2=0x7f2b9c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/2784179971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b9c0047c0 0x7f2b9c004c40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009ac0 tx=0x7f2b9c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bb8f86640 1 -- 172.21.15.50:0/2784179971 reap_dead start 2023-10-19T10:37:40.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bba9e9640 1 -- 172.21.15.50:0/2784179971 shutdown_connections 2023-10-19T10:37:40.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bba9e9640 1 -- 172.21.15.50:0/2784179971 >> 172.21.15.50:0/2784179971 conn(0x7f2bb409b970 msgr2=0x7f2bb40d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:40.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bba9e9640 1 -- 172.21.15.50:0/2784179971 shutdown_connections 2023-10-19T10:37:40.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.592+0000 7f2bba9e9640 1 -- 172.21.15.50:0/2784179971 wait complete. 2023-10-19T10:37:40.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 Processor -- start 2023-10-19T10:37:40.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 -- start start 2023-10-19T10:37:40.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c0047c0 0x7f2bb41558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40c7900 0x7f2bb40c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 0x7f2bb41e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40c6560 con 0x7f2bb415b620 2023-10-19T10:37:40.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4156170 con 0x7f2b9c0047c0 2023-10-19T10:37:40.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.594+0000 7f2bba9e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40c61c0 con 0x7f2bb40c7900 2023-10-19T10:37:40.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 0x7f2bb41e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 0x7f2bb41e6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56818/0 (socket says 172.21.15.50:56818) 2023-10-19T10:37:40.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 -- 172.21.15.50:0/1508249578 learned_addr learned my addr 172.21.15.50:0/1508249578 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:40.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb37fe640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40c7900 0x7f2bb40c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb3fff640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c0047c0 0x7f2bb41558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 -- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40c7900 msgr2=0x7f2bb40c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40c7900 0x7f2bb40c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 -- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c0047c0 msgr2=0x7f2bb41558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c0047c0 0x7f2bb41558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 -- 172.21.15.50:0/1508249578 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40c8070 con 0x7f2bb415b620 2023-10-19T10:37:40.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb8f86640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 0x7f2bb41e6ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4161320 tx=0x7f2bac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00eb90 con 0x7f2bb415b620 2023-10-19T10:37:40.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00ecf0 con 0x7f2bb415b620 2023-10-19T10:37:40.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb40c8350 con 0x7f2bb415b620 2023-10-19T10:37:40.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac014670 con 0x7f2bb415b620 2023-10-19T10:37:40.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.595+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb40c8630 con 0x7f2bb415b620 2023-10-19T10:37:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.596+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb40d09b0 con 0x7f2bb415b620 2023-10-19T10:37:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.597+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2bac0147d0 con 0x7f2bb415b620 2023-10-19T10:37:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.597+0000 7f2bb0ff9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ba006a050 0x7f2ba006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.597+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2bac0884a0 con 0x7f2bb415b620 2023-10-19T10:37:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.597+0000 7f2bb3fff640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ba006a050 0x7f2ba006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.598+0000 7f2bb3fff640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ba006a050 0x7f2ba006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2bb415bfa0 tx=0x7f2ba8009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.601+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2bac0539a0 con 0x7f2bb415b620 2023-10-19T10:37:40.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.778+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bb406b690 con 0x7f2bb415b620 2023-10-19T10:37:40.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.779+0000 7f2bb0ff9640 1 -- 172.21.15.50:0/1508249578 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bac056fc0 con 0x7f2bb415b620 2023-10-19T10:37:40.782 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:40.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ba006a050 msgr2=0x7f2ba006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ba006a050 0x7f2ba006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2bb415bfa0 tx=0x7f2ba8009110 comp rx=0 tx=0).stop 2023-10-19T10:37:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 msgr2=0x7f2bb41e6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 0x7f2bb41e6ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4161320 tx=0x7f2bac00e530 comp rx=0 tx=0).stop 2023-10-19T10:37:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 shutdown_connections 2023-10-19T10:37:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ba006a050 0x7f2ba006c510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bb415b620 0x7f2bb41e6ae0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bb40c7900 0x7f2bb40c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 --2- 172.21.15.50:0/1508249578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b9c0047c0 0x7f2bb41558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.780+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 >> 172.21.15.50:0/1508249578 conn(0x7f2bb409b970 msgr2=0x7f2bb406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:40.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.781+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 shutdown_connections 2023-10-19T10:37:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.781+0000 7f2bba9e9640 1 -- 172.21.15.50:0/1508249578 wait complete. 2023-10-19T10:37:40.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:40.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.900+0000 7f081d5f7640 1 Processor -- start 2023-10-19T10:37:40.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.900+0000 7f081d5f7640 1 -- start start 2023-10-19T10:37:40.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f081d5f7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f081815c510 0x7f08180cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f081d5f7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f08180b10b0 0x7f0818160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f081d5f7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180b7ca0 con 0x7f08180d23b0 2023-10-19T10:37:40.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f081d5f7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08180d21c0 con 0x7f08180b10b0 2023-10-19T10:37:40.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f081d5f7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08180b7ef0 con 0x7f081815c510 2023-10-19T10:37:40.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f08180b10b0 0x7f0818160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f08180b10b0 0x7f0818160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56822/0 (socket says 172.21.15.50:56822) 2023-10-19T10:37:40.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 -- 172.21.15.50:0/1388643163 learned_addr learned my addr 172.21.15.50:0/1388643163 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:40.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 -- 172.21.15.50:0/1388643163 >> v2:172.21.15.183:3300/0 conn(0x7f081815c510 msgr2=0x7f08180cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 --2- 172.21.15.50:0/1388643163 >> v2:172.21.15.183:3300/0 conn(0x7f081815c510 0x7f08180cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 -- 172.21.15.50:0/1388643163 >> v1:172.21.15.183:6789/0 conn(0x7f08180d23b0 legacy=0x7f08180d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:40.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.901+0000 7f08167fc640 1 -- 172.21.15.50:0/1388643163 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180b81d0 con 0x7f08180b10b0 2023-10-19T10:37:40.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.902+0000 7f08167fc640 1 --2- 172.21.15.50:0/1388643163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180b10b0 0x7f0818160bd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08180cfcc0 tx=0x7f080002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52e66d8fe301af server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.902+0000 7f0815ffb640 1 -- 172.21.15.50:0/1388643163 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800031870 con 0x7f08180b10b0 2023-10-19T10:37:40.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.902+0000 7f0815ffb640 1 -- 172.21.15.50:0/1388643163 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000319d0 con 0x7f08180b10b0 2023-10-19T10:37:40.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.902+0000 7f0815ffb640 1 -- 172.21.15.50:0/1388643163 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800031d60 con 0x7f08180b10b0 2023-10-19T10:37:40.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 -- 172.21.15.50:0/1388643163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180b10b0 msgr2=0x7f0818160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 --2- 172.21.15.50:0/1388643163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180b10b0 0x7f0818160bd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08180cfcc0 tx=0x7f080002b910 comp rx=0 tx=0).stop 2023-10-19T10:37:40.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 -- 172.21.15.50:0/1388643163 shutdown_connections 2023-10-19T10:37:40.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 --2- 172.21.15.50:0/1388643163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180b10b0 0x7f0818160bd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 --2- 172.21.15.50:0/1388643163 >> v2:172.21.15.183:3300/0 conn(0x7f081815c510 0x7f08180cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 -- 172.21.15.50:0/1388643163 >> 172.21.15.50:0/1388643163 conn(0x7f0818071320 msgr2=0x7f0818071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 -- 172.21.15.50:0/1388643163 shutdown_connections 2023-10-19T10:37:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.903+0000 7f081d5f7640 1 -- 172.21.15.50:0/1388643163 wait complete. 2023-10-19T10:37:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.904+0000 7f081d5f7640 1 Processor -- start 2023-10-19T10:37:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.904+0000 7f081d5f7640 1 -- start start 2023-10-19T10:37:40.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f081d5f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08180b10b0 0x7f08180caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f081d5f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 0x7f0818155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f081d5f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f081815c510 0x7f08181f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f081d5f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180ca540 con 0x7f08180d23b0 2023-10-19T10:37:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f081d5f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180cb490 con 0x7f08180b10b0 2023-10-19T10:37:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f081d5f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08181f29f0 con 0x7f081815c510 2023-10-19T10:37:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 0x7f0818155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 0x7f0818155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56834/0 (socket says 172.21.15.50:56834) 2023-10-19T10:37:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 -- 172.21.15.50:0/3944798027 learned_addr learned my addr 172.21.15.50:0/3944798027 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 -- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f081815c510 msgr2=0x7f08181f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f081815c510 0x7f08181f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f0816ffd640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08180b10b0 0x7f08180caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 -- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08180b10b0 msgr2=0x7f08180caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:40.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08180b10b0 0x7f08180caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:40.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f08167fc640 1 -- 172.21.15.50:0/3944798027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000094d0 con 0x7f08180d23b0 2023-10-19T10:37:40.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.905+0000 7f0816ffd640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08180b10b0 0x7f08180caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:40.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.906+0000 7f08167fc640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 0x7f0818155880 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f08000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.906+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080003ad10 con 0x7f08180d23b0 2023-10-19T10:37:40.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.906+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080003ae70 con 0x7f08180d23b0 2023-10-19T10:37:40.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.906+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08181f2cd0 con 0x7f08180d23b0 2023-10-19T10:37:40.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.906+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800044610 con 0x7f08180d23b0 2023-10-19T10:37:40.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.906+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08180caa30 con 0x7f08180d23b0 2023-10-19T10:37:40.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.907+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08181f2e60 con 0x7f08180d23b0 2023-10-19T10:37:40.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.907+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f080004d450 con 0x7f08180d23b0 2023-10-19T10:37:40.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.908+0000 7f080f7fe640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f07f806a050 0x7f07f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:40.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.908+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f080002d040 con 0x7f08180d23b0 2023-10-19T10:37:40.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.908+0000 7f0816ffd640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f07f806a050 0x7f07f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:40.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.908+0000 7f0816ffd640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f07f806a050 0x7f07f806c510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08180d16d0 tx=0x7f0810008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:40.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:40.912+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f08000763f0 con 0x7f08180d23b0 2023-10-19T10:37:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.087+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f081804e420 con 0x7f08180d23b0 2023-10-19T10:37:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.087+0000 7f080f7fe640 1 -- 172.21.15.50:0/3944798027 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0800079a10 con 0x7f08180d23b0 2023-10-19T10:37:41.091 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f07f806a050 msgr2=0x7f07f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f07f806a050 0x7f07f806c510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08180d16d0 tx=0x7f0810008040 comp rx=0 tx=0).stop 2023-10-19T10:37:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 msgr2=0x7f0818155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 0x7f0818155880 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f08000046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 shutdown_connections 2023-10-19T10:37:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f07f806a050 0x7f07f806c510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f081815c510 0x7f08181f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08180d23b0 0x7f0818155880 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 --2- 172.21.15.50:0/3944798027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08180b10b0 0x7f08180caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.089+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 >> 172.21.15.50:0/3944798027 conn(0x7f0818071320 msgr2=0x7f0818071700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:41.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.090+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 shutdown_connections 2023-10-19T10:37:41.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.090+0000 7f081d5f7640 1 -- 172.21.15.50:0/3944798027 wait complete. 2023-10-19T10:37:41.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:41.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.209+0000 7f2881db2640 1 Processor -- start 2023-10-19T10:37:41.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 -- start start 2023-10-19T10:37:41.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f287c0b21d0 0x7f287c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f287c0b25b0 0x7f287c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c06f540 0x7f287c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0cfc90 con 0x7f287c0b21d0 2023-10-19T10:37:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c15efb0 con 0x7f287c0b25b0 2023-10-19T10:37:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f2881db2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c15e3a0 con 0x7f287c06f540 2023-10-19T10:37:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f287b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f287c0b21d0 0x7f287c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f287b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f287c0b21d0 0x7f287c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56850/0 (socket says 172.21.15.50:56850) 2023-10-19T10:37:41.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.210+0000 7f287b7fe640 1 -- 172.21.15.50:0/877938037 learned_addr learned my addr 172.21.15.50:0/877938037 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:41.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287affd640 1 --2- 172.21.15.50:0/877938037 >> v2:172.21.15.183:3300/0 conn(0x7f287c0b25b0 0x7f287c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287bfff640 1 --2- 172.21.15.50:0/877938037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c06f540 0x7f287c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287b7fe640 1 -- 172.21.15.50:0/877938037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c06f540 msgr2=0x7f287c06f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287b7fe640 1 --2- 172.21.15.50:0/877938037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c06f540 0x7f287c06f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287b7fe640 1 -- 172.21.15.50:0/877938037 >> v2:172.21.15.183:3300/0 conn(0x7f287c0b25b0 msgr2=0x7f287c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287b7fe640 1 --2- 172.21.15.50:0/877938037 >> v2:172.21.15.183:3300/0 conn(0x7f287c0b25b0 0x7f287c0cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.211+0000 7f287b7fe640 1 -- 172.21.15.50:0/877938037 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c15e680 con 0x7f287c0b21d0 2023-10-19T10:37:41.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f287b7fe640 1 --2- 172.21.15.50:0/877938037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c0ce020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f287c070640 tx=0x7f286c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fc0dff3606d718 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f287a7fc640 1 -- 172.21.15.50:0/877938037 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c031870 con 0x7f287c0b21d0 2023-10-19T10:37:41.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f287a7fc640 1 -- 172.21.15.50:0/877938037 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c0319d0 con 0x7f287c0b21d0 2023-10-19T10:37:41.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f287a7fc640 1 -- 172.21.15.50:0/877938037 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c031d60 con 0x7f287c0b21d0 2023-10-19T10:37:41.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 -- 172.21.15.50:0/877938037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 msgr2=0x7f287c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 --2- 172.21.15.50:0/877938037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c0ce020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f287c070640 tx=0x7f286c02b910 comp rx=0 tx=0).stop 2023-10-19T10:37:41.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 -- 172.21.15.50:0/877938037 shutdown_connections 2023-10-19T10:37:41.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 --2- 172.21.15.50:0/877938037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c06f540 0x7f287c06f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 --2- 172.21.15.50:0/877938037 >> v2:172.21.15.183:3300/0 conn(0x7f287c0b25b0 0x7f287c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 --2- 172.21.15.50:0/877938037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c0ce020 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 -- 172.21.15.50:0/877938037 >> 172.21.15.50:0/877938037 conn(0x7f287c0ccf60 msgr2=0x7f287c0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:41.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 -- 172.21.15.50:0/877938037 shutdown_connections 2023-10-19T10:37:41.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.212+0000 7f2881db2640 1 -- 172.21.15.50:0/877938037 wait complete. 2023-10-19T10:37:41.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.213+0000 7f2881db2640 1 Processor -- start 2023-10-19T10:37:41.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 -- start start 2023-10-19T10:37:41.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f287c06f540 0x7f287c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c0b25b0 0x7f287c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0b0b70 con 0x7f287c0b21d0 2023-10-19T10:37:41.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c8f90 con 0x7f287c06f540 2023-10-19T10:37:41.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f2881db2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c9270 con 0x7f287c0b25b0 2023-10-19T10:37:41.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.214+0000 7f287affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56858/0 (socket says 172.21.15.50:56858) 2023-10-19T10:37:41.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 -- 172.21.15.50:0/1629095663 learned_addr learned my addr 172.21.15.50:0/1629095663 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:41.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287bfff640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c0b25b0 0x7f287c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287b7fe640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f287c06f540 0x7f287c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 -- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c0b25b0 msgr2=0x7f287c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c0b25b0 0x7f287c15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 -- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f287c06f540 msgr2=0x7f287c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f287c06f540 0x7f287c0c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 -- 172.21.15.50:0/1629095663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c0094d0 con 0x7f287c0b21d0 2023-10-19T10:37:41.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287bfff640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c0b25b0 0x7f287c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:41.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f287affd640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c15cc50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f287000e9b0 tx=0x7f287000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870009670 con 0x7f287c0b21d0 2023-10-19T10:37:41.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287000c9a0 con 0x7f287c0b21d0 2023-10-19T10:37:41.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287000cd50 con 0x7f287c0b21d0 2023-10-19T10:37:41.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c0c9550 con 0x7f287c0b21d0 2023-10-19T10:37:41.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.215+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c0c9b10 con 0x7f287c0b21d0 2023-10-19T10:37:41.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.216+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c04e420 con 0x7f287c0b21d0 2023-10-19T10:37:41.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.217+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2870013470 con 0x7f287c0b21d0 2023-10-19T10:37:41.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.217+0000 7f2863fff640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f286806a050 0x7f286806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.217+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2870088ae0 con 0x7f287c0b21d0 2023-10-19T10:37:41.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.217+0000 7f287b7fe640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f286806a050 0x7f286806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.218+0000 7f287b7fe640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f286806a050 0x7f286806c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f286c036000 tx=0x7f286c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.222+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2870053f30 con 0x7f287c0b21d0 2023-10-19T10:37:41.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.396+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f287c0cfe50 con 0x7f287c0b21d0 2023-10-19T10:37:41.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.397+0000 7f2863fff640 1 -- 172.21.15.50:0/1629095663 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2870057550 con 0x7f287c0b21d0 2023-10-19T10:37:41.400 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:41.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f286806a050 msgr2=0x7f286806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f286806a050 0x7f286806c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f286c036000 tx=0x7f286c03c040 comp rx=0 tx=0).stop 2023-10-19T10:37:41.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 msgr2=0x7f287c15cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c15cc50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f287000e9b0 tx=0x7f287000ee70 comp rx=0 tx=0).stop 2023-10-19T10:37:41.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 shutdown_connections 2023-10-19T10:37:41.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f286806a050 0x7f286806c510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f287c0b25b0 0x7f287c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f287c0b21d0 0x7f287c15cc50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 --2- 172.21.15.50:0/1629095663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f287c06f540 0x7f287c0c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.398+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 >> 172.21.15.50:0/1629095663 conn(0x7f287c0ccf60 msgr2=0x7f287c06f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:41.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.399+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 shutdown_connections 2023-10-19T10:37:41.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.399+0000 7f2881db2640 1 -- 172.21.15.50:0/1629095663 wait complete. 2023-10-19T10:37:41.412 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:41.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.516+0000 7fbf5bcbd640 1 Processor -- start 2023-10-19T10:37:41.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.516+0000 7fbf5bcbd640 1 -- start start 2023-10-19T10:37:41.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5bcbd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c570 0x7fbf540d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5bcbd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbf5415c950 0x7fbf540cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5bcbd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbf5406b760 0x7fbf540d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5bcbd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf540cde60 con 0x7fbf5415c950 2023-10-19T10:37:41.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5bcbd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5406aa30 con 0x7fbf5406b760 2023-10-19T10:37:41.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5bcbd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5406b2d0 con 0x7fbf5415c570 2023-10-19T10:37:41.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbf5415c950 0x7fbf540cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbf5415c950 0x7fbf540cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56874/0 (socket says 172.21.15.50:56874) 2023-10-19T10:37:41.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 -- 172.21.15.50:0/4096782813 learned_addr learned my addr 172.21.15.50:0/4096782813 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:41.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf5a25a640 1 --2- 172.21.15.50:0/4096782813 >> v2:172.21.15.183:3300/0 conn(0x7fbf5406b760 0x7fbf540d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59a59640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c570 0x7fbf540d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 -- 172.21.15.50:0/4096782813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c570 msgr2=0x7fbf540d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:41.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c570 0x7fbf540d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 -- 172.21.15.50:0/4096782813 >> v2:172.21.15.183:3300/0 conn(0x7fbf5406b760 msgr2=0x7fbf540d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:41.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 --2- 172.21.15.50:0/4096782813 >> v2:172.21.15.183:3300/0 conn(0x7fbf5406b760 0x7fbf540d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59258640 1 -- 172.21.15.50:0/4096782813 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540ce090 con 0x7fbf5415c950 2023-10-19T10:37:41.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.517+0000 7fbf59a59640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c570 0x7fbf540d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:41.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.518+0000 7fbf59258640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c950 0x7fbf540cd920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf541615b0 tx=0x7fbf4802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6098219bc56558e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.518+0000 7fbf58a57640 1 -- 172.21.15.50:0/4096782813 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480317d0 con 0x7fbf5415c950 2023-10-19T10:37:41.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.518+0000 7fbf58a57640 1 -- 172.21.15.50:0/4096782813 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf48031930 con 0x7fbf5415c950 2023-10-19T10:37:41.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.518+0000 7fbf58a57640 1 -- 172.21.15.50:0/4096782813 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48031cc0 con 0x7fbf5415c950 2023-10-19T10:37:41.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.518+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/4096782813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c950 msgr2=0x7fbf540cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.518+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c950 0x7fbf540cd920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf541615b0 tx=0x7fbf4802b870 comp rx=0 tx=0).stop 2023-10-19T10:37:41.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/4096782813 shutdown_connections 2023-10-19T10:37:41.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/4096782813 >> v2:172.21.15.183:3300/0 conn(0x7fbf5406b760 0x7fbf540d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c950 0x7fbf540cd920 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/4096782813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c570 0x7fbf540d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/4096782813 >> 172.21.15.50:0/4096782813 conn(0x7fbf54089ff0 msgr2=0x7fbf5409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:41.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/4096782813 shutdown_connections 2023-10-19T10:37:41.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.519+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/4096782813 wait complete. 2023-10-19T10:37:41.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.520+0000 7fbf5bcbd640 1 Processor -- start 2023-10-19T10:37:41.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.520+0000 7fbf5bcbd640 1 -- start start 2023-10-19T10:37:41.529 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5406b760 0x7fbf540b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 0x7fbf540afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c950 0x7fbf540b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c6240 con 0x7fbf5415c570 2023-10-19T10:37:41.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf541526f0 con 0x7fbf5406b760 2023-10-19T10:37:41.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf541f1fc0 con 0x7fbf5415c950 2023-10-19T10:37:41.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 0x7fbf540afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 0x7fbf540afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56878/0 (socket says 172.21.15.50:56878) 2023-10-19T10:37:41.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 -- 172.21.15.50:0/920404007 learned_addr learned my addr 172.21.15.50:0/920404007 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:41.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59a59640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5406b760 0x7fbf540b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5a25a640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c950 0x7fbf540b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 -- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c950 msgr2=0x7fbf540b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c950 0x7fbf540b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 -- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5406b760 msgr2=0x7fbf540b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5406b760 0x7fbf540b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 -- 172.21.15.50:0/920404007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf480094d0 con 0x7fbf5415c570 2023-10-19T10:37:41.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf59258640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 0x7fbf540afec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf4802b8b0 tx=0x7fbf480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48032870 con 0x7fbf5415c570 2023-10-19T10:37:41.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf48032a10 con 0x7fbf5415c570 2023-10-19T10:37:41.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf541f22a0 con 0x7fbf5415c570 2023-10-19T10:37:41.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48032dc0 con 0x7fbf5415c570 2023-10-19T10:37:41.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.521+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf541f2860 con 0x7fbf5415c570 2023-10-19T10:37:41.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.522+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf541f2a50 con 0x7fbf5415c570 2023-10-19T10:37:41.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.523+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbf48030070 con 0x7fbf5415c570 2023-10-19T10:37:41.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.523+0000 7fbf527fc640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbf4006a050 0x7fbf4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.523+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbf480aa6e0 con 0x7fbf5415c570 2023-10-19T10:37:41.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.524+0000 7fbf59a59640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbf4006a050 0x7fbf4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.524+0000 7fbf59a59640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbf4006a050 0x7fbf4006c510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf5415b3c0 tx=0x7fbf4c00e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.528+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbf48075c10 con 0x7fbf5415c570 2023-10-19T10:37:41.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.705+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf540729a0 con 0x7fbf5415c570 2023-10-19T10:37:41.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.707+0000 7fbf527fc640 1 -- 172.21.15.50:0/920404007 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fbf48079230 con 0x7fbf5415c570 2023-10-19T10:37:41.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:41.710 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:41.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbf4006a050 msgr2=0x7fbf4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbf4006a050 0x7fbf4006c510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf5415b3c0 tx=0x7fbf4c00e040 comp rx=0 tx=0).stop 2023-10-19T10:37:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 msgr2=0x7fbf540afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 0x7fbf540afec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf4802b8b0 tx=0x7fbf480046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 shutdown_connections 2023-10-19T10:37:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbf4006a050 0x7fbf4006c510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5415c950 0x7fbf540b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf5415c570 0x7fbf540afec0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 --2- 172.21.15.50:0/920404007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5406b760 0x7fbf540b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 >> 172.21.15.50:0/920404007 conn(0x7fbf54089ff0 msgr2=0x7fbf5415bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:41.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 shutdown_connections 2023-10-19T10:37:41.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.708+0000 7fbf5bcbd640 1 -- 172.21.15.50:0/920404007 wait complete. 2023-10-19T10:37:41.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:41.723 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.829+0000 7f1b0e3cb640 1 Processor -- start 2023-10-19T10:37:41.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.829+0000 7f1b0e3cb640 1 -- start start 2023-10-19T10:37:41.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b0e3cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0815a650 0x7f1b0815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b0e3cb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b0806dbe0 0x7f1b080c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b0e3cb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0815ef60 con 0x7f1b0806e040 2023-10-19T10:37:41.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b0e3cb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0815aa30 con 0x7f1b0806dbe0 2023-10-19T10:37:41.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b0e3cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0806e5c0 con 0x7f1b0815a650 2023-10-19T10:37:41.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b077fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b0806dbe0 0x7f1b080c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b077fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b0806dbe0 0x7f1b080c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56886/0 (socket says 172.21.15.50:56886) 2023-10-19T10:37:41.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b077fe640 1 -- 172.21.15.50:0/2083322755 learned_addr learned my addr 172.21.15.50:0/2083322755 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:41.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.830+0000 7f1b0c968640 1 --1- 172.21.15.50:0/2083322755 >> v1:172.21.15.50:6789/0 conn(0x7f1b0806e040 0x7f1b080b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40636/0 (socket says 172.21.15.50:40636) 2023-10-19T10:37:41.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b07fff640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0815a650 0x7f1b0815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b077fe640 1 -- 172.21.15.50:0/2083322755 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0815a650 msgr2=0x7f1b0815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:41.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b077fe640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0815a650 0x7f1b0815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b077fe640 1 -- 172.21.15.50:0/2083322755 >> v1:172.21.15.50:6789/0 conn(0x7f1b0806e040 legacy=0x7f1b080b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b077fe640 1 -- 172.21.15.50:0/2083322755 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0806e8a0 con 0x7f1b0806dbe0 2023-10-19T10:37:41.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b06ffd640 1 -- 172.21.15.50:0/2083322755 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877993610 0 0) 0x7f1b0815ef60 con 0x7f1b0806e040 2023-10-19T10:37:41.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b07fff640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0815a650 0x7f1b0815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:41.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b077fe640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806dbe0 0x7f1b080c97c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009870 tx=0x7f1af802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8e6af9abfc96c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.831+0000 7f1b06ffd640 1 -- 172.21.15.50:0/2083322755 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8031640 con 0x7f1b0806dbe0 2023-10-19T10:37:41.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b06ffd640 1 -- 172.21.15.50:0/2083322755 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af80317a0 con 0x7f1b0806dbe0 2023-10-19T10:37:41.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b06ffd640 1 -- 172.21.15.50:0/2083322755 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8031b30 con 0x7f1b0806dbe0 2023-10-19T10:37:41.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2083322755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806dbe0 msgr2=0x7f1b080c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806dbe0 0x7f1b080c97c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009870 tx=0x7f1af802b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2083322755 shutdown_connections 2023-10-19T10:37:41.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806dbe0 0x7f1b080c97c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2083322755 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0815a650 0x7f1b0815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2083322755 >> 172.21.15.50:0/2083322755 conn(0x7f1b080cf170 msgr2=0x7f1b080cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2083322755 shutdown_connections 2023-10-19T10:37:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.832+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2083322755 wait complete. 2023-10-19T10:37:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.833+0000 7f1b0e3cb640 1 Processor -- start 2023-10-19T10:37:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.833+0000 7f1b0e3cb640 1 -- start start 2023-10-19T10:37:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0e3cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0806dbe0 0x7f1b0815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0e3cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 0x7f1b08151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0e3cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b0815a650 0x7f1b08154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0e3cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b08154060 con 0x7f1b0806e040 2023-10-19T10:37:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0e3cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b080a7b50 con 0x7f1b0815a650 2023-10-19T10:37:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0e3cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b080a7870 con 0x7f1b0806dbe0 2023-10-19T10:37:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 0x7f1b08151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 0x7f1b08151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56896/0 (socket says 172.21.15.50:56896) 2023-10-19T10:37:41.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b077fe640 1 -- 172.21.15.50:0/2340518362 learned_addr learned my addr 172.21.15.50:0/2340518362 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:41.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b0c968640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b0815a650 0x7f1b08154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.834+0000 7f1b07fff640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0806dbe0 0x7f1b0815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b077fe640 1 -- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0806dbe0 msgr2=0x7f1b0815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b077fe640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0806dbe0 0x7f1b0815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b077fe640 1 -- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b0815a650 msgr2=0x7f1b08154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:41.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b077fe640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b0815a650 0x7f1b08154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:41.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b077fe640 1 -- 172.21.15.50:0/2340518362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80094d0 con 0x7f1b0806e040 2023-10-19T10:37:41.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b07fff640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0806dbe0 0x7f1b0815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:41.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b077fe640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 0x7f1b08151750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1af8000c00 tx=0x7f1af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af803ac60 con 0x7f1b0806e040 2023-10-19T10:37:41.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af803ade0 con 0x7f1b0806e040 2023-10-19T10:37:41.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8044630 con 0x7f1b0806e040 2023-10-19T10:37:41.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b081539a0 con 0x7f1b0806e040 2023-10-19T10:37:41.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.835+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b081549c0 con 0x7f1b0806e040 2023-10-19T10:37:41.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.836+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b08153800 con 0x7f1b0806e040 2023-10-19T10:37:41.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.837+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1af802f050 con 0x7f1b0806e040 2023-10-19T10:37:41.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.837+0000 7f1b04ff9640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1ae806a050 0x7f1ae806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:41.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.837+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1af80aa910 con 0x7f1b0806e040 2023-10-19T10:37:41.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.837+0000 7f1b07fff640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1ae806a050 0x7f1ae806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:41.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.838+0000 7f1b07fff640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1ae806a050 0x7f1ae806c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b08158e30 tx=0x7f1af0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:41.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:41.841+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1b08153800 con 0x7f1b0806e040 2023-10-19T10:37:42.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.016+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b081541a0 con 0x7f1b0806e040 2023-10-19T10:37:42.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.016+0000 7f1b04ff9640 1 -- 172.21.15.50:0/2340518362 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b081541a0 con 0x7f1b0806e040 2023-10-19T10:37:42.020 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:42.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1ae806a050 msgr2=0x7f1ae806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1ae806a050 0x7f1ae806c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b08158e30 tx=0x7f1af0008040 comp rx=0 tx=0).stop 2023-10-19T10:37:42.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 msgr2=0x7f1b08151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 0x7f1b08151750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1af8000c00 tx=0x7f1af80046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 shutdown_connections 2023-10-19T10:37:42.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1ae806a050 0x7f1ae806c510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b0815a650 0x7f1b08154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b0806e040 0x7f1b08151750 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 --2- 172.21.15.50:0/2340518362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b0806dbe0 0x7f1b0815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 >> 172.21.15.50:0/2340518362 conn(0x7f1b080cf170 msgr2=0x7f1b081595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 shutdown_connections 2023-10-19T10:37:42.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.018+0000 7f1b0e3cb640 1 -- 172.21.15.50:0/2340518362 wait complete. 2023-10-19T10:37:42.031 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:42.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.138+0000 7f830191f640 1 Processor -- start 2023-10-19T10:37:42.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.138+0000 7f830191f640 1 -- start start 2023-10-19T10:37:42.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f830191f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f82fc15c660 0x7f82fc06a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f830191f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc070780 con 0x7f82fc0cd460 2023-10-19T10:37:42.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f830191f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0cca10 con 0x7f82fc160240 2023-10-19T10:37:42.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f830191f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc06a750 con 0x7f82fc15c660 2023-10-19T10:37:42.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f82faffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f82fc15c660 0x7f82fc06a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f82fb7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f82fc0cd460 0x7f82fc070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40640/0 (socket says 172.21.15.50:40640) 2023-10-19T10:37:42.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.139+0000 7f82fb7fe640 1 -- 172.21.15.50:0/1023934058 learned_addr learned my addr 172.21.15.50:0/1023934058 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:42.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.140+0000 7f82faffd640 1 -- 172.21.15.50:0/1023934058 >> v1:172.21.15.183:6789/0 conn(0x7f82fc160240 legacy=0x7f82fc0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.140+0000 7f82faffd640 1 -- 172.21.15.50:0/1023934058 >> v1:172.21.15.50:6789/0 conn(0x7f82fc0cd460 legacy=0x7f82fc070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.140+0000 7f82faffd640 1 -- 172.21.15.50:0/1023934058 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc160720 con 0x7f82fc15c660 2023-10-19T10:37:42.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.140+0000 7f82faffd640 1 --2- 172.21.15.50:0/1023934058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc15c660 0x7f82fc06a210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097a0 tx=0x7f82f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8575ef846daa5e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f82f9ffb640 1 -- 172.21.15.50:0/1023934058 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4030610 con 0x7f82fc15c660 2023-10-19T10:37:42.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f82f9ffb640 1 -- 172.21.15.50:0/1023934058 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4030770 con 0x7f82fc15c660 2023-10-19T10:37:42.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f82f9ffb640 1 -- 172.21.15.50:0/1023934058 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4030b00 con 0x7f82fc15c660 2023-10-19T10:37:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 -- 172.21.15.50:0/1023934058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc15c660 msgr2=0x7f82fc06a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 --2- 172.21.15.50:0/1023934058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc15c660 0x7f82fc06a210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097a0 tx=0x7f82f402b960 comp rx=0 tx=0).stop 2023-10-19T10:37:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 -- 172.21.15.50:0/1023934058 shutdown_connections 2023-10-19T10:37:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 --2- 172.21.15.50:0/1023934058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc15c660 0x7f82fc06a210 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 -- 172.21.15.50:0/1023934058 >> 172.21.15.50:0/1023934058 conn(0x7f82fc0cbc00 msgr2=0x7f82fc0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 -- 172.21.15.50:0/1023934058 shutdown_connections 2023-10-19T10:37:42.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.141+0000 7f830191f640 1 -- 172.21.15.50:0/1023934058 wait complete. 2023-10-19T10:37:42.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.142+0000 7f830191f640 1 Processor -- start 2023-10-19T10:37:42.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 -- start start 2023-10-19T10:37:42.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 0x7f82fc0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82fc15c660 0x7f82fc0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82fc160240 0x7f82fc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0b30d0 con 0x7f82fc0cd460 2023-10-19T10:37:42.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0b24c0 con 0x7f82fc15c660 2023-10-19T10:37:42.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f830191f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0b27a0 con 0x7f82fc160240 2023-10-19T10:37:42.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f82faffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 0x7f82fc0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f82faffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 0x7f82fc0b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56910/0 (socket says 172.21.15.50:56910) 2023-10-19T10:37:42.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.143+0000 7f82faffd640 1 -- 172.21.15.50:0/3636682275 learned_addr learned my addr 172.21.15.50:0/3636682275 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:42.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82faffd640 1 -- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82fc160240 msgr2=0x7f82fc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82faffd640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82fc160240 0x7f82fc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82faffd640 1 -- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82fc15c660 msgr2=0x7f82fc0ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82faffd640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82fc15c660 0x7f82fc0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82faffd640 1 -- 172.21.15.50:0/3636682275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4009400 con 0x7f82fc0cd460 2023-10-19T10:37:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82faffd640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 0x7f82fc0b1dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098c0 tx=0x7f82f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f403e070 con 0x7f82fc0cd460 2023-10-19T10:37:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4039a10 con 0x7f82fc0cd460 2023-10-19T10:37:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4039da0 con 0x7f82fc0cd460 2023-10-19T10:37:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc0b2a80 con 0x7f82fc0cd460 2023-10-19T10:37:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.144+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc0ad7e0 con 0x7f82fc0cd460 2023-10-19T10:37:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.145+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc0cb900 con 0x7f82fc0cd460 2023-10-19T10:37:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.145+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f82f404b400 con 0x7f82fc0cd460 2023-10-19T10:37:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.146+0000 7f82f3fff640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82dc06a050 0x7f82dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.146+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f82f40aadc0 con 0x7f82fc0cd460 2023-10-19T10:37:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.146+0000 7f82fa7fc640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82dc06a050 0x7f82dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.147+0000 7f82fa7fc640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82dc06a050 0x7f82dc06c510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000f60 tx=0x7f82e40079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.150+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f82f4076210 con 0x7f82fc0cd460 2023-10-19T10:37:42.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.327+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f82fc04e420 con 0x7f82fc0cd460 2023-10-19T10:37:42.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.328+0000 7f82f3fff640 1 -- 172.21.15.50:0/3636682275 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f82f40340a0 con 0x7f82fc0cd460 2023-10-19T10:37:42.331 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:42.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82dc06a050 msgr2=0x7f82dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82dc06a050 0x7f82dc06c510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000f60 tx=0x7f82e40079b0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 msgr2=0x7f82fc0b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 0x7f82fc0b1dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098c0 tx=0x7f82f40046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 shutdown_connections 2023-10-19T10:37:42.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82dc06a050 0x7f82dc06c510 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82fc160240 0x7f82fc0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82fc15c660 0x7f82fc0ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 --2- 172.21.15.50:0/3636682275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82fc0cd460 0x7f82fc0b1dc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 >> 172.21.15.50:0/3636682275 conn(0x7f82fc0cbc00 msgr2=0x7f82fc06c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 shutdown_connections 2023-10-19T10:37:42.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.330+0000 7f830191f640 1 -- 172.21.15.50:0/3636682275 wait complete. 2023-10-19T10:37:42.343 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:42.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.450+0000 7f8a0071f640 1 Processor -- start 2023-10-19T10:37:42.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.450+0000 7f8a0071f640 1 -- start start 2023-10-19T10:37:42.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f8a0071f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f89f806add0 0x7f89f81650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f8a0071f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f8069930 con 0x7f89f806cf80 2023-10-19T10:37:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f8a0071f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f806c7b0 con 0x7f89f815a430 2023-10-19T10:37:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f8a0071f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f806d8a0 con 0x7f89f806add0 2023-10-19T10:37:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fdcba640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f89f806cf80 0x7f89f806c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40650/0 (socket says 172.21.15.50:40650) 2023-10-19T10:37:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fdcba640 1 -- 172.21.15.50:0/410694006 learned_addr learned my addr 172.21.15.50:0/410694006 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468774986 0 0) 0x7f89f8069930 con 0x7f89f806cf80 2023-10-19T10:37:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e4003520 con 0x7f89f806cf80 2023-10-19T10:37:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fecbc640 1 --2- 172.21.15.50:0/410694006 >> v2:172.21.15.183:3300/0 conn(0x7f89f806add0 0x7f89f81650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753145064 0 0) 0x7f89e4003520 con 0x7f89f806cf80 2023-10-19T10:37:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 >> v2:172.21.15.183:3300/0 conn(0x7f89f806add0 msgr2=0x7f89f81650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 --2- 172.21.15.50:0/410694006 >> v2:172.21.15.183:3300/0 conn(0x7f89f806add0 0x7f89f81650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 >> v1:172.21.15.183:6789/0 conn(0x7f89f815a430 legacy=0x7f89f806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f806d5c0 con 0x7f89f806cf80 2023-10-19T10:37:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f89f4001670 con 0x7f89f806cf80 2023-10-19T10:37:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.451+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 >> v1:172.21.15.50:6789/0 conn(0x7f89f806cf80 legacy=0x7f89f806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fd4b9640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89e4003970 0x7f89e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fd4b9640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89e40047c0 0x7f89e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fd4b9640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f806d8a0 con 0x7f89e4005180 2023-10-19T10:37:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c020 con 0x7f89e4003970 2023-10-19T10:37:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c1e0 con 0x7f89e40047c0 2023-10-19T10:37:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fecbc640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fe4bb640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89e4003970 0x7f89e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fdcba640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89e40047c0 0x7f89e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fecbc640 1 -- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89e40047c0 msgr2=0x7f89e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:42.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fecbc640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89e40047c0 0x7f89e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fecbc640 1 -- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89e4003970 msgr2=0x7f89e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:42.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fecbc640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89e4003970 0x7f89e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fecbc640 1 -- 172.21.15.50:0/410694006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4003520 con 0x7f89e4005180 2023-10-19T10:37:42.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fe4bb640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89e4003970 0x7f89e4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:42.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.452+0000 7f89fdcba640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89e40047c0 0x7f89e4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:42.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f89fecbc640 1 -- 172.21.15.50:0/410694006 reap_dead start 2023-10-19T10:37:42.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f89fecbc640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89e400b980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fc0 tx=0x7f89e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08c9385382319da server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f002cc60 con 0x7f89e4005180 2023-10-19T10:37:42.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f000f040 con 0x7f89e4005180 2023-10-19T10:37:42.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f89fd4b9640 1 -- 172.21.15.50:0/410694006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0003990 con 0x7f89e4005180 2023-10-19T10:37:42.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f8a0071f640 1 -- 172.21.15.50:0/410694006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 msgr2=0x7f89e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f8a0071f640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89e400b980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fc0 tx=0x7f89e40057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f8a0071f640 1 -- 172.21.15.50:0/410694006 shutdown_connections 2023-10-19T10:37:42.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f8a0071f640 1 --2- 172.21.15.50:0/410694006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89e400b980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f8a0071f640 1 -- 172.21.15.50:0/410694006 >> 172.21.15.50:0/410694006 conn(0x7f89f80b03f0 msgr2=0x7f89f80b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.453+0000 7f8a0071f640 1 -- 172.21.15.50:0/410694006 shutdown_connections 2023-10-19T10:37:42.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.454+0000 7f8a0071f640 1 -- 172.21.15.50:0/410694006 wait complete. 2023-10-19T10:37:42.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 Processor -- start 2023-10-19T10:37:42.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 -- start start 2023-10-19T10:37:42.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89f8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89f81e2230 0x7f89f80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89f81e26b0 0x7f89f80b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f81e3790 con 0x7f89e4005180 2023-10-19T10:37:42.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8151c70 con 0x7f89f81e26b0 2023-10-19T10:37:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f8a0071f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f81e2c10 con 0x7f89f81e2230 2023-10-19T10:37:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.455+0000 7f89fe4bb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89f8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89f8151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56916/0 (socket says 172.21.15.50:56916) 2023-10-19T10:37:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 -- 172.21.15.50:0/2611314928 learned_addr learned my addr 172.21.15.50:0/2611314928 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:42.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fdcba640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89f81e2230 0x7f89f80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fecbc640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89f81e26b0 0x7f89f80b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 -- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89f81e2230 msgr2=0x7f89f80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89f81e2230 0x7f89f80b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 -- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89f81e26b0 msgr2=0x7f89f80b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89f81e26b0 0x7f89f80b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 -- 172.21.15.50:0/2611314928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f81e2ef0 con 0x7f89e4005180 2023-10-19T10:37:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fecbc640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89f81e26b0 0x7f89f80b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fdcba640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89f81e2230 0x7f89f80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89fe4bb640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89f8151730 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89e8000c00 tx=0x7f89e800e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80052e0 con 0x7f89e4005180 2023-10-19T10:37:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8005440 con 0x7f89e4005180 2023-10-19T10:37:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f81e31d0 con 0x7f89e4005180 2023-10-19T10:37:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8003830 con 0x7f89e4005180 2023-10-19T10:37:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.456+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f8153450 con 0x7f89e4005180 2023-10-19T10:37:42.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.457+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f8153670 con 0x7f89e4005180 2023-10-19T10:37:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.457+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f89e8003990 con 0x7f89e4005180 2023-10-19T10:37:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.458+0000 7f89eeffd640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89dc06a050 0x7f89dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.458+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f89e8087ff0 con 0x7f89e4005180 2023-10-19T10:37:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.458+0000 7f89fdcba640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89dc06a050 0x7f89dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.459+0000 7f89fdcba640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89dc06a050 0x7f89dc06c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f89f4001200 tx=0x7f89f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.462+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f89f8153670 con 0x7f89e4005180 2023-10-19T10:37:42.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.636+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89f8153670 con 0x7f89e4005180 2023-10-19T10:37:42.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.636+0000 7f89eeffd640 1 -- 172.21.15.50:0/2611314928 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89f8153670 con 0x7f89e4005180 2023-10-19T10:37:42.639 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:42.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89dc06a050 msgr2=0x7f89dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89dc06a050 0x7f89dc06c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f89f4001200 tx=0x7f89f4009040 comp rx=0 tx=0).stop 2023-10-19T10:37:42.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 msgr2=0x7f89f8151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89f8151730 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89e8000c00 tx=0x7f89e800e8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 shutdown_connections 2023-10-19T10:37:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f89dc06a050 0x7f89dc06c510 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89f81e26b0 0x7f89f80b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89f81e2230 0x7f89f80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 --2- 172.21.15.50:0/2611314928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89e4005180 0x7f89f8151730 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 >> 172.21.15.50:0/2611314928 conn(0x7f89f80b03f0 msgr2=0x7f89f80b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 shutdown_connections 2023-10-19T10:37:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.638+0000 7f8a0071f640 1 -- 172.21.15.50:0/2611314928 wait complete. 2023-10-19T10:37:42.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:42.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 Processor -- start 2023-10-19T10:37:42.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 -- start start 2023-10-19T10:37:42.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1c1c0b4070 0x7f1c1c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1c0b4990 con 0x7f1c1c15cdf0 2023-10-19T10:37:42.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c15d280 con 0x7f1c1c0b4070 2023-10-19T10:37:42.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.759+0000 7f1c23199640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c15d560 con 0x7f1c1c15c960 2023-10-19T10:37:42.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1c1c0b4070 0x7f1c1c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1c1c0b4070 0x7f1c1c0b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56930/0 (socket says 172.21.15.50:56930) 2023-10-19T10:37:42.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 -- 172.21.15.50:0/219149059 learned_addr learned my addr 172.21.15.50:0/219149059 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:42.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c1bfff640 1 --2- 172.21.15.50:0/219149059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 -- 172.21.15.50:0/219149059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 msgr2=0x7f1c1c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:42.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 --2- 172.21.15.50:0/219149059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 -- 172.21.15.50:0/219149059 >> v1:172.21.15.183:6789/0 conn(0x7f1c1c15cdf0 legacy=0x7f1c1c160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.760+0000 7f1c20f35640 1 -- 172.21.15.50:0/219149059 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c15d840 con 0x7f1c1c0b4070 2023-10-19T10:37:42.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c20f35640 1 --2- 172.21.15.50:0/219149059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c0b4450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009850 tx=0x7f1c0c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be59445a33cae21a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c1b7fe640 1 -- 172.21.15.50:0/219149059 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c004230 con 0x7f1c1c0b4070 2023-10-19T10:37:42.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c1b7fe640 1 -- 172.21.15.50:0/219149059 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c004390 con 0x7f1c1c0b4070 2023-10-19T10:37:42.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c1b7fe640 1 -- 172.21.15.50:0/219149059 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c003810 con 0x7f1c1c0b4070 2023-10-19T10:37:42.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 -- 172.21.15.50:0/219149059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b4070 msgr2=0x7f1c1c0b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 --2- 172.21.15.50:0/219149059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c0b4450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009850 tx=0x7f1c0c02b960 comp rx=0 tx=0).stop 2023-10-19T10:37:42.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 -- 172.21.15.50:0/219149059 shutdown_connections 2023-10-19T10:37:42.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 --2- 172.21.15.50:0/219149059 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 --2- 172.21.15.50:0/219149059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c0b4450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 -- 172.21.15.50:0/219149059 >> 172.21.15.50:0/219149059 conn(0x7f1c1c09b970 msgr2=0x7f1c1c0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 -- 172.21.15.50:0/219149059 shutdown_connections 2023-10-19T10:37:42.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.761+0000 7f1c23199640 1 -- 172.21.15.50:0/219149059 wait complete. 2023-10-19T10:37:42.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.762+0000 7f1c23199640 1 Processor -- start 2023-10-19T10:37:42.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 -- start start 2023-10-19T10:37:42.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 0x7f1c1c0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0c69e0 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0c6cc0 con 0x7f1c1c0b4070 2023-10-19T10:37:42.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c23199640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0c6fa0 con 0x7f1c1c15c960 2023-10-19T10:37:42.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.763+0000 7f1c21736640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 0x7f1c1c0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 0x7f1c1c0b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56940/0 (socket says 172.21.15.50:56940) 2023-10-19T10:37:42.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 -- 172.21.15.50:0/3294208664 learned_addr learned my addr 172.21.15.50:0/3294208664 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:42.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c1bfff640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c20f35640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 -- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 msgr2=0x7f1c1c0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 -- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c0b4070 msgr2=0x7f1c1c150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 -- 172.21.15.50:0/3294208664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009460 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c21736640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 0x7f1c1c0b0170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400c730 tx=0x7f1c1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14003b90 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c14003cf0 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14005310 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c0c7280 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.764+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c151130 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.765+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c0c77c0 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.766+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c14005680 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.766+0000 7f1c197fa640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0806a050 0x7f1c0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:42.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.766+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1c140888b0 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.766+0000 7f1c20f35640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0806a050 0x7f1c0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:42.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.767+0000 7f1c20f35640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0806a050 0x7f1c0806c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c035000 tx=0x7f1c0c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:42.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.770+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1c14053bd0 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.944+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c1c04e420 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.944+0000 7f1c197fa640 1 -- 172.21.15.50:0/3294208664 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c140571f0 con 0x7f1c1c0b7f10 2023-10-19T10:37:42.947 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:42.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0806a050 msgr2=0x7f1c0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0806a050 0x7f1c0806c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c035000 tx=0x7f1c0c008030 comp rx=0 tx=0).stop 2023-10-19T10:37:42.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 msgr2=0x7f1c1c0b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:42.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 0x7f1c1c0b0170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400c730 tx=0x7f1c1400cbf0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 shutdown_connections 2023-10-19T10:37:42.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1c0806a050 0x7f1c0806c510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c0b7f10 0x7f1c1c0b0170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c1c15c960 0x7f1c1c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 --2- 172.21.15.50:0/3294208664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c0b4070 0x7f1c1c150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:42.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 >> 172.21.15.50:0/3294208664 conn(0x7f1c1c09b970 msgr2=0x7f1c1c0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:42.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 shutdown_connections 2023-10-19T10:37:42.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:42.946+0000 7f1c23199640 1 -- 172.21.15.50:0/3294208664 wait complete. 2023-10-19T10:37:42.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:43.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.066+0000 7f887dc36640 1 Processor -- start 2023-10-19T10:37:43.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.066+0000 7f887dc36640 1 -- start start 2023-10-19T10:37:43.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f887dc36640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b38f0 0x7f88780d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f887dc36640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8878161210 0x7f88780b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f887dc36640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780af250 con 0x7f88780b3090 2023-10-19T10:37:43.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f887dc36640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88780b0190 con 0x7f8878161210 2023-10-19T10:37:43.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f887dc36640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780af510 con 0x7f88780b38f0 2023-10-19T10:37:43.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8878161210 0x7f88780b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8878161210 0x7f88780b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56944/0 (socket says 172.21.15.50:56944) 2023-10-19T10:37:43.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 -- 172.21.15.50:0/1963894303 learned_addr learned my addr 172.21.15.50:0/1963894303 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:43.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f88777fe640 1 --2- 172.21.15.50:0/1963894303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b38f0 0x7f88780d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 -- 172.21.15.50:0/1963894303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b38f0 msgr2=0x7f88780d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:43.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 --2- 172.21.15.50:0/1963894303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b38f0 0x7f88780d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 -- 172.21.15.50:0/1963894303 >> v1:172.21.15.183:6789/0 conn(0x7f88780b3090 legacy=0x7f8878160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:43.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.067+0000 7f8877fff640 1 -- 172.21.15.50:0/1963894303 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780af7f0 con 0x7f8878161210 2023-10-19T10:37:43.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.068+0000 7f8877fff640 1 --2- 172.21.15.50:0/1963894303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878161210 0x7f88780b05b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8870009e50 tx=0x7f887002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d3f70530ada192 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.068+0000 7f88767fc640 1 -- 172.21.15.50:0/1963894303 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887002ca70 con 0x7f8878161210 2023-10-19T10:37:43.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.068+0000 7f88767fc640 1 -- 172.21.15.50:0/1963894303 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887002cbd0 con 0x7f8878161210 2023-10-19T10:37:43.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.068+0000 7f88767fc640 1 -- 172.21.15.50:0/1963894303 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88700343f0 con 0x7f8878161210 2023-10-19T10:37:43.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 -- 172.21.15.50:0/1963894303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878161210 msgr2=0x7f88780b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 --2- 172.21.15.50:0/1963894303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878161210 0x7f88780b05b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8870009e50 tx=0x7f887002bed0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 -- 172.21.15.50:0/1963894303 shutdown_connections 2023-10-19T10:37:43.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 --2- 172.21.15.50:0/1963894303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878161210 0x7f88780b05b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 --2- 172.21.15.50:0/1963894303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b38f0 0x7f88780d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 -- 172.21.15.50:0/1963894303 >> 172.21.15.50:0/1963894303 conn(0x7f887809b960 msgr2=0x7f88780d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:43.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 -- 172.21.15.50:0/1963894303 shutdown_connections 2023-10-19T10:37:43.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.069+0000 7f887dc36640 1 -- 172.21.15.50:0/1963894303 wait complete. 2023-10-19T10:37:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.070+0000 7f887dc36640 1 Processor -- start 2023-10-19T10:37:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.070+0000 7f887dc36640 1 -- start start 2023-10-19T10:37:43.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f887dc36640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b3090 0x7f88781f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f887dc36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 0x7f88781f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f887dc36640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878161210 0x7f88780cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f887dc36640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887815b110 con 0x7f88780b38f0 2023-10-19T10:37:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f887dc36640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88781548d0 con 0x7f8878161210 2023-10-19T10:37:43.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f887dc36640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88781f65c0 con 0x7f88780b3090 2023-10-19T10:37:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 0x7f88781f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 0x7f88781f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56956/0 (socket says 172.21.15.50:56956) 2023-10-19T10:37:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 -- 172.21.15.50:0/2285125684 learned_addr learned my addr 172.21.15.50:0/2285125684 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f88777fe640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b3090 0x7f88781f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8877fff640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878161210 0x7f88780cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 -- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b3090 msgr2=0x7f88781f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b3090 0x7f88781f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 -- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878161210 msgr2=0x7f88780cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878161210 0x7f88780cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8876ffd640 1 -- 172.21.15.50:0/2285125684 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870009b30 con 0x7f88780b38f0 2023-10-19T10:37:43.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f88777fe640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b3090 0x7f88781f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:43.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.071+0000 7f8877fff640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878161210 0x7f88780cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:43.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.072+0000 7f8876ffd640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 0x7f88781f1a10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c400 tx=0x7f886c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.072+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c011790 con 0x7f88780b38f0 2023-10-19T10:37:43.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.072+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c011930 con 0x7f88780b38f0 2023-10-19T10:37:43.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.072+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88780c9d10 con 0x7f88780b38f0 2023-10-19T10:37:43.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.072+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c011ce0 con 0x7f88780b38f0 2023-10-19T10:37:43.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.072+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88781f1f50 con 0x7f88780b38f0 2023-10-19T10:37:43.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.073+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88780c9ea0 con 0x7f88780b38f0 2023-10-19T10:37:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.073+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f886c01c460 con 0x7f88780b38f0 2023-10-19T10:37:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.074+0000 7f885ffff640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f886406a050 0x7f886406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.074+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f886c088800 con 0x7f88780b38f0 2023-10-19T10:37:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.074+0000 7f88777fe640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f886406a050 0x7f886406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.074+0000 7f88777fe640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f886406a050 0x7f886406c510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8868006fd0 tx=0x7f8868008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.078+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f886c053bc0 con 0x7f88780b38f0 2023-10-19T10:37:43.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.255+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f887804e420 con 0x7f88780b38f0 2023-10-19T10:37:43.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.255+0000 7f885ffff640 1 -- 172.21.15.50:0/2285125684 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f886c0571e0 con 0x7f88780b38f0 2023-10-19T10:37:43.258 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:43.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f886406a050 msgr2=0x7f886406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f886406a050 0x7f886406c510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8868006fd0 tx=0x7f8868008040 comp rx=0 tx=0).stop 2023-10-19T10:37:43.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 msgr2=0x7f88781f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 0x7f88781f1a10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c400 tx=0x7f886c00c8c0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 shutdown_connections 2023-10-19T10:37:43.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f886406a050 0x7f886406c510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878161210 0x7f88780cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88780b38f0 0x7f88781f1a10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 --2- 172.21.15.50:0/2285125684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88780b3090 0x7f88781f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 >> 172.21.15.50:0/2285125684 conn(0x7f887809b960 msgr2=0x7f88780b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:43.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 shutdown_connections 2023-10-19T10:37:43.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.257+0000 7f887dc36640 1 -- 172.21.15.50:0/2285125684 wait complete. 2023-10-19T10:37:43.270 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:43.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.374+0000 7fa9369c7640 1 Processor -- start 2023-10-19T10:37:43.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.374+0000 7fa9369c7640 1 -- start start 2023-10-19T10:37:43.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa9369c7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa9300d2db0 0x7fa93015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa9369c7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93006aa30 con 0x7fa93015c570 2023-10-19T10:37:43.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa9369c7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93006b2d0 con 0x7fa9300d0e30 2023-10-19T10:37:43.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa9369c7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93015ce90 con 0x7fa9300d2db0 2023-10-19T10:37:43.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa93015c570 0x7fa93006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40660/0 (socket says 172.21.15.50:40660) 2023-10-19T10:37:43.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92ffff640 1 -- 172.21.15.50:0/169932274 learned_addr learned my addr 172.21.15.50:0/169932274 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:43.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa934f64640 1 --2- 172.21.15.50:0/169932274 >> v2:172.21.15.183:3300/0 conn(0x7fa9300d2db0 0x7fa93015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858730503 0 0) 0x7fa93006aa30 con 0x7fa93015c570 2023-10-19T10:37:43.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa918003520 con 0x7fa93015c570 2023-10-19T10:37:43.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1624634047 0 0) 0x7fa918003520 con 0x7fa93015c570 2023-10-19T10:37:43.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 >> v2:172.21.15.183:3300/0 conn(0x7fa9300d2db0 msgr2=0x7fa93015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:43.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 --2- 172.21.15.50:0/169932274 >> v2:172.21.15.183:3300/0 conn(0x7fa9300d2db0 0x7fa93015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 >> v1:172.21.15.183:6789/0 conn(0x7fa9300d0e30 legacy=0x7fa9300d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300d1270 con 0x7fa93015c570 2023-10-19T10:37:43.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fa9200017f0 con 0x7fa93015c570 2023-10-19T10:37:43.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 >> v1:172.21.15.50:6789/0 conn(0x7fa93015c570 legacy=0x7fa93006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa918003970 0x7fa918003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9180047c0 0x7fa918004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa918005180 0x7fa91800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93015ce90 con 0x7fa9180047c0 2023-10-19T10:37:43.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91800c120 con 0x7fa918005180 2023-10-19T10:37:43.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.375+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91800c2e0 con 0x7fa918003970 2023-10-19T10:37:43.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92f7fe640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9180047c0 0x7fa918004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92ffff640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa918003970 0x7fa918003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa934f64640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa918005180 0x7fa91800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92f7fe640 1 -- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa918003970 msgr2=0x7fa918003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:43.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92f7fe640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa918003970 0x7fa918003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92f7fe640 1 -- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa918005180 msgr2=0x7fa91800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:43.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92f7fe640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa918005180 0x7fa91800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa92f7fe640 1 -- 172.21.15.50:0/169932274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918003520 con 0x7fa9180047c0 2023-10-19T10:37:43.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.376+0000 7fa934f64640 1 -- 172.21.15.50:0/169932274 reap_dead start 2023-10-19T10:37:43.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa92f7fe640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9180047c0 0x7fa918004c40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa924001750 tx=0x7fa9180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95c342d235cd6eb server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240052d0 con 0x7fa9180047c0 2023-10-19T10:37:43.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92402e050 con 0x7fa9180047c0 2023-10-19T10:37:43.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa92effd640 1 -- 172.21.15.50:0/169932274 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924003830 con 0x7fa9180047c0 2023-10-19T10:37:43.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 -- 172.21.15.50:0/169932274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9180047c0 msgr2=0x7fa918004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9180047c0 0x7fa918004c40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa924001750 tx=0x7fa9180057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 -- 172.21.15.50:0/169932274 shutdown_connections 2023-10-19T10:37:43.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 --2- 172.21.15.50:0/169932274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9180047c0 0x7fa918004c40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 -- 172.21.15.50:0/169932274 >> 172.21.15.50:0/169932274 conn(0x7fa930089ff0 msgr2=0x7fa93009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:43.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 -- 172.21.15.50:0/169932274 shutdown_connections 2023-10-19T10:37:43.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.377+0000 7fa9369c7640 1 -- 172.21.15.50:0/169932274 wait complete. 2023-10-19T10:37:43.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 Processor -- start 2023-10-19T10:37:43.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 -- start start 2023-10-19T10:37:43.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9180047c0 0x7fa93015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa930159a70 0x7fa9300cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 0x7fa9300c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300c6bd0 con 0x7fa9300c6210 2023-10-19T10:37:43.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93015e7c0 con 0x7fa9180047c0 2023-10-19T10:37:43.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.379+0000 7fa9369c7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93015eaa0 con 0x7fa930159a70 2023-10-19T10:37:43.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 0x7fa9300c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 0x7fa9300c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56970/0 (socket says 172.21.15.50:56970) 2023-10-19T10:37:43.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 -- 172.21.15.50:0/4013814577 learned_addr learned my addr 172.21.15.50:0/4013814577 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:43.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa92f7fe640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa930159a70 0x7fa9300cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa92ffff640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9180047c0 0x7fa93015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 -- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa930159a70 msgr2=0x7fa9300cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa930159a70 0x7fa9300cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 -- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9180047c0 msgr2=0x7fa93015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9180047c0 0x7fa93015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 -- 172.21.15.50:0/4013814577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93015ed80 con 0x7fa9300c6210 2023-10-19T10:37:43.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa934f64640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 0x7fa9300c6690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9300c61e0 tx=0x7fa92800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800e040 con 0x7fa9300c6210 2023-10-19T10:37:43.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92800f040 con 0x7fa9300c6210 2023-10-19T10:37:43.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800fec0 con 0x7fa9300c6210 2023-10-19T10:37:43.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93015f060 con 0x7fa9300c6210 2023-10-19T10:37:43.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.380+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9300cc4d0 con 0x7fa9300c6210 2023-10-19T10:37:43.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.381+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa930073990 con 0x7fa9300c6210 2023-10-19T10:37:43.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.382+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa928015430 con 0x7fa9300c6210 2023-10-19T10:37:43.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.382+0000 7fa92cff9640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa91c06a050 0x7fa91c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.382+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa928088f70 con 0x7fa9300c6210 2023-10-19T10:37:43.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.382+0000 7fa92ffff640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa91c06a050 0x7fa91c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.383+0000 7fa92ffff640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa91c06a050 0x7fa91c06c510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa920001e10 tx=0x7fa920009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.387+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa928057e70 con 0x7fa9300c6210 2023-10-19T10:37:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.562+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9300d3480 con 0x7fa9300c6210 2023-10-19T10:37:43.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.563+0000 7fa92cff9640 1 -- 172.21.15.50:0/4013814577 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fa928057a00 con 0x7fa9300c6210 2023-10-19T10:37:43.566 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:43.567 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:43.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa91c06a050 msgr2=0x7fa91c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa91c06a050 0x7fa91c06c510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa920001e10 tx=0x7fa920009040 comp rx=0 tx=0).stop 2023-10-19T10:37:43.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 msgr2=0x7fa9300c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 0x7fa9300c6690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9300c61e0 tx=0x7fa92800f1b0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 shutdown_connections 2023-10-19T10:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa91c06a050 0x7fa91c06c510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9300c6210 0x7fa9300c6690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa930159a70 0x7fa9300cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 --2- 172.21.15.50:0/4013814577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9180047c0 0x7fa93015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 >> 172.21.15.50:0/4013814577 conn(0x7fa930089ff0 msgr2=0x7fa93015b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:43.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 shutdown_connections 2023-10-19T10:37:43.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.565+0000 7fa9369c7640 1 -- 172.21.15.50:0/4013814577 wait complete. 2023-10-19T10:37:43.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:43.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:43.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 Processor -- start 2023-10-19T10:37:43.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 -- start start 2023-10-19T10:37:43.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f894c15cdf0 0x7f894c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c15d280 con 0x7f894c0b4070 2023-10-19T10:37:43.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c15d560 con 0x7f894c15cdf0 2023-10-19T10:37:43.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.687+0000 7f895217f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c15d840 con 0x7f894c15c960 2023-10-19T10:37:43.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894b7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f894c0b4070 0x7f894c0b4450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40676/0 (socket says 172.21.15.50:40676) 2023-10-19T10:37:43.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894b7fe640 1 -- 172.21.15.50:0/1091398702 learned_addr learned my addr 172.21.15.50:0/1091398702 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:43.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894bfff640 1 --2- 172.21.15.50:0/1091398702 >> v2:172.21.15.183:3300/0 conn(0x7f894c15cdf0 0x7f894c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894affd640 1 --2- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894a7fc640 1 -- 172.21.15.50:0/1091398702 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907170322 0 0) 0x7f894c15d280 con 0x7f894c0b4070 2023-10-19T10:37:43.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894a7fc640 1 -- 172.21.15.50:0/1091398702 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8934003520 con 0x7f894c0b4070 2023-10-19T10:37:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894bfff640 1 -- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 msgr2=0x7f894c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894bfff640 1 --2- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894bfff640 1 -- 172.21.15.50:0/1091398702 >> v1:172.21.15.50:6789/0 conn(0x7f894c0b4070 legacy=0x7f894c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.688+0000 7f894bfff640 1 -- 172.21.15.50:0/1091398702 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c15db20 con 0x7f894c15cdf0 2023-10-19T10:37:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.689+0000 7f894bfff640 1 --2- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15cdf0 0x7f894c0cd4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8944009d90 tx=0x7f894402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ded39b5eb1d2dbc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.689+0000 7f894a7fc640 1 -- 172.21.15.50:0/1091398702 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894402cc50 con 0x7f894c15cdf0 2023-10-19T10:37:43.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.689+0000 7f894a7fc640 1 -- 172.21.15.50:0/1091398702 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894402cdb0 con 0x7f894c15cdf0 2023-10-19T10:37:43.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.689+0000 7f894a7fc640 1 -- 172.21.15.50:0/1091398702 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944034660 con 0x7f894c15cdf0 2023-10-19T10:37:43.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 -- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15cdf0 msgr2=0x7f894c0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 --2- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15cdf0 0x7f894c0cd4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8944009d90 tx=0x7f894402bea0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 -- 172.21.15.50:0/1091398702 shutdown_connections 2023-10-19T10:37:43.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 --2- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15cdf0 0x7f894c0cd4b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 --2- 172.21.15.50:0/1091398702 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 -- 172.21.15.50:0/1091398702 >> 172.21.15.50:0/1091398702 conn(0x7f894c09b970 msgr2=0x7f894c0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:43.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 -- 172.21.15.50:0/1091398702 shutdown_connections 2023-10-19T10:37:43.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.690+0000 7f895217f640 1 -- 172.21.15.50:0/1091398702 wait complete. 2023-10-19T10:37:43.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.691+0000 7f895217f640 1 Processor -- start 2023-10-19T10:37:43.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.691+0000 7f895217f640 1 -- start start 2023-10-19T10:37:43.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f895217f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 0x7f894c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f895217f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f895217f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c0c84b0 0x7f894c15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f895217f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0af2c0 con 0x7f894c0b4070 2023-10-19T10:37:43.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f895217f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0c8ae0 con 0x7f894c0c84b0 2023-10-19T10:37:43.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f895217f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0c7880 con 0x7f894c15c960 2023-10-19T10:37:43.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 0x7f894c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 0x7f894c158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56978/0 (socket says 172.21.15.50:56978) 2023-10-19T10:37:43.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 -- 172.21.15.50:0/144961923 learned_addr learned my addr 172.21.15.50:0/144961923 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:43.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894bfff640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c0c84b0 0x7f894c15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894affd640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 -- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 msgr2=0x7f894c0c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 -- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c0c84b0 msgr2=0x7f894c15f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c0c84b0 0x7f894c15f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 -- 172.21.15.50:0/144961923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944009a70 con 0x7f894c0b4070 2023-10-19T10:37:43.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f894b7fe640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 0x7f894c158680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c700 tx=0x7f893c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.692+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c007270 con 0x7f894c0b4070 2023-10-19T10:37:43.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.693+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c0073d0 con 0x7f894c0b4070 2023-10-19T10:37:43.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.693+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c0af560 con 0x7f894c0b4070 2023-10-19T10:37:43.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.693+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c0057a0 con 0x7f894c0b4070 2023-10-19T10:37:43.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.693+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c0afb20 con 0x7f894c0b4070 2023-10-19T10:37:43.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.693+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c0afd10 con 0x7f894c0b4070 2023-10-19T10:37:43.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.694+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f893c005900 con 0x7f894c0b4070 2023-10-19T10:37:43.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.694+0000 7f8933fff640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f893806a050 0x7f893806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:43.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.695+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f893c087de0 con 0x7f894c0b4070 2023-10-19T10:37:43.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.695+0000 7f894affd640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f893806a050 0x7f893806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:43.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.695+0000 7f894affd640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f893806a050 0x7f893806c510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f894c15ff70 tx=0x7f8940009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:43.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.699+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f893c053310 con 0x7f894c0b4070 2023-10-19T10:37:43.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.879+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f894c04e420 con 0x7f894c0b4070 2023-10-19T10:37:43.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.879+0000 7f8933fff640 1 -- 172.21.15.50:0/144961923 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f893c017020 con 0x7f894c0b4070 2023-10-19T10:37:43.882 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:43.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f893806a050 msgr2=0x7f893806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f893806a050 0x7f893806c510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f894c15ff70 tx=0x7f8940009190 comp rx=0 tx=0).stop 2023-10-19T10:37:43.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 msgr2=0x7f894c158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:43.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 0x7f894c158680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c700 tx=0x7f893c00cbc0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 shutdown_connections 2023-10-19T10:37:43.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f893806a050 0x7f893806c510 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c0c84b0 0x7f894c15f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c15c960 0x7f894c0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 --2- 172.21.15.50:0/144961923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f894c0b4070 0x7f894c158680 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:43.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 >> 172.21.15.50:0/144961923 conn(0x7f894c09b970 msgr2=0x7f894c0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:43.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 shutdown_connections 2023-10-19T10:37:43.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:43.881+0000 7f895217f640 1 -- 172.21.15.50:0/144961923 wait complete. 2023-10-19T10:37:43.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:44.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.001+0000 7fa76ec92640 1 Processor -- start 2023-10-19T10:37:44.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ec92640 1 -- start start 2023-10-19T10:37:44.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ec92640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa7680b25b0 0x7fa7680ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ec92640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa76815b500 0x7fa7680cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ec92640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76815b990 con 0x7fa7680b21d0 2023-10-19T10:37:44.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ec92640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7680cfc90 con 0x7fa7680b25b0 2023-10-19T10:37:44.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ec92640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76815efb0 con 0x7fa76815b500 2023-10-19T10:37:44.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ca2e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa7680b21d0 0x7fa76806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40692/0 (socket says 172.21.15.50:40692) 2023-10-19T10:37:44.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.002+0000 7fa76ca2e640 1 -- 172.21.15.50:0/2179290622 learned_addr learned my addr 172.21.15.50:0/2179290622 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:44.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2179290622 >> v2:172.21.15.183:3300/0 conn(0x7fa7680b25b0 0x7fa7680ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa76d22f640 1 --2- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa76815b500 0x7fa7680cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75f7fe640 1 -- 172.21.15.50:0/2179290622 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818631593 0 0) 0x7fa76815b990 con 0x7fa7680b21d0 2023-10-19T10:37:44.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75f7fe640 1 -- 172.21.15.50:0/2179290622 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa750003520 con 0x7fa7680b21d0 2023-10-19T10:37:44.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75ffff640 1 -- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa76815b500 msgr2=0x7fa7680cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa76815b500 0x7fa7680cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75ffff640 1 -- 172.21.15.50:0/2179290622 >> v1:172.21.15.50:6789/0 conn(0x7fa7680b21d0 legacy=0x7fa76806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.003+0000 7fa75ffff640 1 -- 172.21.15.50:0/2179290622 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76815e3a0 con 0x7fa7680b25b0 2023-10-19T10:37:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7680b25b0 0x7fa7680ce020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097a0 tx=0x7fa75802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89a75cca6f162ee server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa75f7fe640 1 -- 172.21.15.50:0/2179290622 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758031830 con 0x7fa7680b25b0 2023-10-19T10:37:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa75f7fe640 1 -- 172.21.15.50:0/2179290622 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa758031990 con 0x7fa7680b25b0 2023-10-19T10:37:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa75f7fe640 1 -- 172.21.15.50:0/2179290622 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758031d20 con 0x7fa7680b25b0 2023-10-19T10:37:44.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa76ec92640 1 -- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7680b25b0 msgr2=0x7fa7680ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7680b25b0 0x7fa7680ce020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097a0 tx=0x7fa75802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa76ec92640 1 -- 172.21.15.50:0/2179290622 shutdown_connections 2023-10-19T10:37:44.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa76815b500 0x7fa7680cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2179290622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7680b25b0 0x7fa7680ce020 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.004+0000 7fa76ec92640 1 -- 172.21.15.50:0/2179290622 >> 172.21.15.50:0/2179290622 conn(0x7fa7680ccf60 msgr2=0x7fa7680cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.005+0000 7fa76ec92640 1 -- 172.21.15.50:0/2179290622 shutdown_connections 2023-10-19T10:37:44.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.005+0000 7fa76ec92640 1 -- 172.21.15.50:0/2179290622 wait complete. 2023-10-19T10:37:44.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.006+0000 7fa76ec92640 1 Processor -- start 2023-10-19T10:37:44.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.006+0000 7fa76ec92640 1 -- start start 2023-10-19T10:37:44.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.006+0000 7fa76ec92640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 0x7fa7681f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.006+0000 7fa76ec92640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7680b25b0 0x7fa7680c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ec92640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76815b500 0x7fa7680c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ec92640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680c8f20 con 0x7fa7680b21d0 2023-10-19T10:37:44.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ec92640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680c9200 con 0x7fa76815b500 2023-10-19T10:37:44.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ec92640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680c94e0 con 0x7fa7680b25b0 2023-10-19T10:37:44.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 0x7fa7681f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 0x7fa7681f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56986/0 (socket says 172.21.15.50:56986) 2023-10-19T10:37:44.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 -- 172.21.15.50:0/2178514479 learned_addr learned my addr 172.21.15.50:0/2178514479 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:44.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7680b25b0 0x7fa7680c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76d22f640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76815b500 0x7fa7680c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 -- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7680b25b0 msgr2=0x7fa7680c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7680b25b0 0x7fa7680c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 -- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76815b500 msgr2=0x7fa7680c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76815b500 0x7fa7680c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 -- 172.21.15.50:0/2178514479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa758009400 con 0x7fa7680b21d0 2023-10-19T10:37:44.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7680b25b0 0x7fa7680c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:44.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ca2e640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 0x7fa7681f16c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c500 tx=0x7fa76400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764005ce0 con 0x7fa7680b21d0 2023-10-19T10:37:44.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764005e40 con 0x7fa7680b21d0 2023-10-19T10:37:44.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7680c97c0 con 0x7fa7680b21d0 2023-10-19T10:37:44.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764007330 con 0x7fa7680b21d0 2023-10-19T10:37:44.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.007+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7681f1c00 con 0x7fa7680b21d0 2023-10-19T10:37:44.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.008+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7680c9ce0 con 0x7fa7680b21d0 2023-10-19T10:37:44.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.009+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa764007490 con 0x7fa7680b21d0 2023-10-19T10:37:44.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.009+0000 7fa75d7fa640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa75406a080 0x7fa75406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.009+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa764087e30 con 0x7fa7680b21d0 2023-10-19T10:37:44.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.010+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa75406a080 0x7fa75406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.010+0000 7fa75ffff640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa75406a080 0x7fa75406c540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa758036000 tx=0x7fa75803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.014+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa7680c9ce0 con 0x7fa7680b21d0 2023-10-19T10:37:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.192+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7680c9ce0 con 0x7fa7680b21d0 2023-10-19T10:37:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.192+0000 7fa75d7fa640 1 -- 172.21.15.50:0/2178514479 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7680c9ce0 con 0x7fa7680b21d0 2023-10-19T10:37:44.195 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.194+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa75406a080 msgr2=0x7fa75406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.194+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa75406a080 0x7fa75406c540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa758036000 tx=0x7fa75803c040 comp rx=0 tx=0).stop 2023-10-19T10:37:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.194+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 msgr2=0x7fa7681f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.194+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 0x7fa7681f16c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c500 tx=0x7fa76400c9c0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 shutdown_connections 2023-10-19T10:37:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa75406a080 0x7fa75406c540 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76815b500 0x7fa7680c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7680b25b0 0x7fa7680c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 --2- 172.21.15.50:0/2178514479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7680b21d0 0x7fa7681f16c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 >> 172.21.15.50:0/2178514479 conn(0x7fa7680ccf60 msgr2=0x7fa7680b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 shutdown_connections 2023-10-19T10:37:44.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.195+0000 7fa76ec92640 1 -- 172.21.15.50:0/2178514479 wait complete. 2023-10-19T10:37:44.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.315+0000 7f4373617640 1 Processor -- start 2023-10-19T10:37:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.315+0000 7f4373617640 1 -- start start 2023-10-19T10:37:44.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4373617640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c0b4070 0x7f436c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4373617640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f436c15c960 0x7f436c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4373617640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c0b4990 con 0x7f436c15cdc0 2023-10-19T10:37:44.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4373617640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c15d280 con 0x7f436c15c960 2023-10-19T10:37:44.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4373617640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c15d560 con 0x7f436c0b4070 2023-10-19T10:37:44.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4370bb2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f436c15c960 0x7f436c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4371bb4640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f436c15cdc0 0x7f436c160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40706/0 (socket says 172.21.15.50:40706) 2023-10-19T10:37:44.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4371bb4640 1 -- 172.21.15.50:0/3644646252 learned_addr learned my addr 172.21.15.50:0/3644646252 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:44.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f43713b3640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c0b4070 0x7f436c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4370bb2640 1 -- 172.21.15.50:0/3644646252 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c0b4070 msgr2=0x7f436c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4370bb2640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c0b4070 0x7f436c0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4370bb2640 1 -- 172.21.15.50:0/3644646252 >> v1:172.21.15.50:6789/0 conn(0x7f436c15cdc0 legacy=0x7f436c160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f4370bb2640 1 -- 172.21.15.50:0/3644646252 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c15d840 con 0x7f436c15c960 2023-10-19T10:37:44.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f436bfff640 1 -- 172.21.15.50:0/3644646252 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223240213 0 0) 0x7f436c0b4990 con 0x7f436c15cdc0 2023-10-19T10:37:44.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.316+0000 7f43713b3640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c0b4070 0x7f436c0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:44.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.317+0000 7f4370bb2640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0d2d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee46fa6932d8c46 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.317+0000 7f436bfff640 1 -- 172.21.15.50:0/3644646252 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360031830 con 0x7f436c15c960 2023-10-19T10:37:44.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.317+0000 7f436bfff640 1 -- 172.21.15.50:0/3644646252 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360031990 con 0x7f436c15c960 2023-10-19T10:37:44.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.317+0000 7f436bfff640 1 -- 172.21.15.50:0/3644646252 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360031d20 con 0x7f436c15c960 2023-10-19T10:37:44.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 -- 172.21.15.50:0/3644646252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 msgr2=0x7f436c0d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0d2d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 -- 172.21.15.50:0/3644646252 shutdown_connections 2023-10-19T10:37:44.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0d2d70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 --2- 172.21.15.50:0/3644646252 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c0b4070 0x7f436c0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 -- 172.21.15.50:0/3644646252 >> 172.21.15.50:0/3644646252 conn(0x7f436c09b970 msgr2=0x7f436c0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 -- 172.21.15.50:0/3644646252 shutdown_connections 2023-10-19T10:37:44.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.318+0000 7f4373617640 1 -- 172.21.15.50:0/3644646252 wait complete. 2023-10-19T10:37:44.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.319+0000 7f4373617640 1 Processor -- start 2023-10-19T10:37:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.319+0000 7f4373617640 1 -- start start 2023-10-19T10:37:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4373617640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0b4070 0x7f436c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4373617640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4373617640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c15cdc0 0x7f436c1f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4373617640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c160740 con 0x7f436c15c960 2023-10-19T10:37:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4373617640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c160bd0 con 0x7f436c0b4070 2023-10-19T10:37:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4373617640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c15fb30 con 0x7f436c15cdc0 2023-10-19T10:37:44.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57008/0 (socket says 172.21.15.50:57008) 2023-10-19T10:37:44.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 -- 172.21.15.50:0/2615037483 learned_addr learned my addr 172.21.15.50:0/2615037483 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:44.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4371bb4640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c15cdc0 0x7f436c1f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f43713b3640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0b4070 0x7f436c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 -- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c15cdc0 msgr2=0x7f436c1f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c15cdc0 0x7f436c1f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 -- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0b4070 msgr2=0x7f436c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0b4070 0x7f436c0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 -- 172.21.15.50:0/2615037483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009400 con 0x7f436c15c960 2023-10-19T10:37:44.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.320+0000 7f4370bb2640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0b0b30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c00 tx=0x7f43600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.321+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360004230 con 0x7f436c15c960 2023-10-19T10:37:44.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.321+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360004390 con 0x7f436c15c960 2023-10-19T10:37:44.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.321+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c15fe10 con 0x7f436c15c960 2023-10-19T10:37:44.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.321+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436003a8a0 con 0x7f436c15c960 2023-10-19T10:37:44.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.321+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c1603d0 con 0x7f436c15c960 2023-10-19T10:37:44.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.322+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c04e420 con 0x7f436c15c960 2023-10-19T10:37:44.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.322+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f436003aa00 con 0x7f436c15c960 2023-10-19T10:37:44.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.323+0000 7f4369ffb640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f435806a050 0x7f435806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.323+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f43600aaf30 con 0x7f436c15c960 2023-10-19T10:37:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.323+0000 7f43713b3640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f435806a050 0x7f435806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.324+0000 7f43713b3640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f435806a050 0x7f435806c510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f436400c8e0 tx=0x7f4364008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.327+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4360076250 con 0x7f436c15c960 2023-10-19T10:37:44.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.502+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f436c160880 con 0x7f436c15c960 2023-10-19T10:37:44.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.503+0000 7f4369ffb640 1 -- 172.21.15.50:0/2615037483 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4360035020 con 0x7f436c15c960 2023-10-19T10:37:44.506 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f435806a050 msgr2=0x7f435806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f435806a050 0x7f435806c510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f436400c8e0 tx=0x7f4364008040 comp rx=0 tx=0).stop 2023-10-19T10:37:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 msgr2=0x7f436c0b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0b0b30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c00 tx=0x7f43600046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 shutdown_connections 2023-10-19T10:37:44.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f435806a050 0x7f435806c510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f436c15cdc0 0x7f436c1f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f436c15c960 0x7f436c0b0b30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 --2- 172.21.15.50:0/2615037483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0b4070 0x7f436c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 >> 172.21.15.50:0/2615037483 conn(0x7f436c09b970 msgr2=0x7f436c15a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 shutdown_connections 2023-10-19T10:37:44.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.505+0000 7f4373617640 1 -- 172.21.15.50:0/2615037483 wait complete. 2023-10-19T10:37:44.518 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:44.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.625+0000 7fe131540640 1 Processor -- start 2023-10-19T10:37:44.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.625+0000 7fe131540640 1 -- start start 2023-10-19T10:37:44.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe131540640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe12c0ca690 0x7fe12c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe131540640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 0x7fe12c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe131540640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c0cde00 con 0x7fe12c06cf00 2023-10-19T10:37:44.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe131540640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12c06d370 con 0x7fe12c0ca690 2023-10-19T10:37:44.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe131540640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c0ccb00 con 0x7fe12c15a310 2023-10-19T10:37:44.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe12b7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe12c06cf00 0x7fe12c0c8d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40720/0 (socket says 172.21.15.50:40720) 2023-10-19T10:37:44.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe12b7fe640 1 -- 172.21.15.50:0/4110860016 learned_addr learned my addr 172.21.15.50:0/4110860016 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:44.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe12affd640 1 --2- 172.21.15.50:0/4110860016 >> v2:172.21.15.183:3300/0 conn(0x7fe12c0ca690 0x7fe12c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 0x7fe12c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39360847 0 0) 0x7fe12c0cde00 con 0x7fe12c06cf00 2023-10-19T10:37:44.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe118003520 con 0x7fe12c06cf00 2023-10-19T10:37:44.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702462390 0 0) 0x7fe118003520 con 0x7fe12c06cf00 2023-10-19T10:37:44.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 msgr2=0x7fe12c06c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.626+0000 7fe129ffb640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 0x7fe12c06c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 >> v2:172.21.15.183:3300/0 conn(0x7fe12c0ca690 msgr2=0x7fe12c0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 --2- 172.21.15.50:0/4110860016 >> v2:172.21.15.183:3300/0 conn(0x7fe12c0ca690 0x7fe12c0cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0ccde0 con 0x7fe12c06cf00 2023-10-19T10:37:44.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fe124001690 con 0x7fe12c06cf00 2023-10-19T10:37:44.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 >> v1:172.21.15.50:6789/0 conn(0x7fe12c06cf00 legacy=0x7fe12c0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 0x7fe118003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe1180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe118004d00 0x7fe118005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c06d370 con 0x7fe118003ef0 2023-10-19T10:37:44.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c0ccb00 con 0x7fe118004d00 2023-10-19T10:37:44.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11800b980 con 0x7fe12c15a310 2023-10-19T10:37:44.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12affd640 1 --2- 172.21.15.50:0/4110860016 >> v2:172.21.15.183:3300/0 conn(0x7fe12c0ca690 0x7fe12c0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:44.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe1180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12b7fe640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe118004d00 0x7fe118005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12affd640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 0x7fe118003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12a7fc640 1 -- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 msgr2=0x7fe118003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c15a310 0x7fe118003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12a7fc640 1 -- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe118004d00 msgr2=0x7fe118005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe118004d00 0x7fe118005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.627+0000 7fe12a7fc640 1 -- 172.21.15.50:0/4110860016 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118003520 con 0x7fe118003ef0 2023-10-19T10:37:44.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe1180047c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe114009aa0 tx=0x7fe1180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a4061c248d755d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11402c6b0 con 0x7fe118003ef0 2023-10-19T10:37:44.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11402c810 con 0x7fe118003ef0 2023-10-19T10:37:44.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe129ffb640 1 -- 172.21.15.50:0/4110860016 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11402cbc0 con 0x7fe118003ef0 2023-10-19T10:37:44.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe131540640 1 -- 172.21.15.50:0/4110860016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 msgr2=0x7fe1180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe131540640 1 --2- 172.21.15.50:0/4110860016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe1180047c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe114009aa0 tx=0x7fe1180057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe12b7fe640 1 -- 172.21.15.50:0/4110860016 reap_dead start 2023-10-19T10:37:44.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe131540640 1 -- 172.21.15.50:0/4110860016 shutdown_connections 2023-10-19T10:37:44.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.628+0000 7fe131540640 1 -- 172.21.15.50:0/4110860016 >> 172.21.15.50:0/4110860016 conn(0x7fe12c0afbe0 msgr2=0x7fe12c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.629+0000 7fe131540640 1 -- 172.21.15.50:0/4110860016 shutdown_connections 2023-10-19T10:37:44.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.629+0000 7fe131540640 1 -- 172.21.15.50:0/4110860016 wait complete. 2023-10-19T10:37:44.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.630+0000 7fe131540640 1 Processor -- start 2023-10-19T10:37:44.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.630+0000 7fe131540640 1 -- start start 2023-10-19T10:37:44.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe12c0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c0ab8e0 0x7fe12c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12c0abd40 0x7fe12c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c155be0 con 0x7fe118003ef0 2023-10-19T10:37:44.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c0ac250 con 0x7fe12c0abd40 2023-10-19T10:37:44.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c0ac530 con 0x7fe12c0ab8e0 2023-10-19T10:37:44.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe12c0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe12c0b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57034/0 (socket says 172.21.15.50:57034) 2023-10-19T10:37:44.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 -- 172.21.15.50:0/1811438695 learned_addr learned my addr 172.21.15.50:0/1811438695 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:44.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c0ab8e0 0x7fe12c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 -- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c0ab8e0 msgr2=0x7fe12c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12b7fe640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12c0abd40 0x7fe12c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c0ab8e0 0x7fe12c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 -- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12c0abd40 msgr2=0x7fe12c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12c0abd40 0x7fe12c06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 -- 172.21.15.50:0/1811438695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0ac810 con 0x7fe118003ef0 2023-10-19T10:37:44.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c0ab8e0 0x7fe12c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:44.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe12affd640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe12c0b70b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe11c011480 tx=0x7fe11c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00e030 con 0x7fe118003ef0 2023-10-19T10:37:44.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c016740 con 0x7fe118003ef0 2023-10-19T10:37:44.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c155e80 con 0x7fe118003ef0 2023-10-19T10:37:44.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.631+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c016ad0 con 0x7fe118003ef0 2023-10-19T10:37:44.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.632+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c1f5970 con 0x7fe118003ef0 2023-10-19T10:37:44.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.632+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c156010 con 0x7fe118003ef0 2023-10-19T10:37:44.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.633+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe11c016c30 con 0x7fe118003ef0 2023-10-19T10:37:44.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.633+0000 7fe1237fe640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe10c06a050 0x7fe10c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.633+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe11c00f070 con 0x7fe118003ef0 2023-10-19T10:37:44.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.634+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe10c06a050 0x7fe10c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.634+0000 7fe12a7fc640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe10c06a050 0x7fe10c06c510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe114036000 tx=0x7fe114030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.638+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe11c08be60 con 0x7fe118003ef0 2023-10-19T10:37:44.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.816+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe12c04e420 con 0x7fe118003ef0 2023-10-19T10:37:44.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.816+0000 7fe1237fe640 1 -- 172.21.15.50:0/1811438695 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe11c054ff0 con 0x7fe118003ef0 2023-10-19T10:37:44.819 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:44.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe10c06a050 msgr2=0x7fe10c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe10c06a050 0x7fe10c06c510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe114036000 tx=0x7fe114030000 comp rx=0 tx=0).stop 2023-10-19T10:37:44.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 msgr2=0x7fe12c0b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe12c0b70b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe11c011480 tx=0x7fe11c011940 comp rx=0 tx=0).stop 2023-10-19T10:37:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 shutdown_connections 2023-10-19T10:37:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe10c06a050 0x7fe10c06c510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12c0abd40 0x7fe12c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe12c0ab8e0 0x7fe12c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 --2- 172.21.15.50:0/1811438695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe118003ef0 0x7fe12c0b70b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 >> 172.21.15.50:0/1811438695 conn(0x7fe12c0afbe0 msgr2=0x7fe12c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 shutdown_connections 2023-10-19T10:37:44.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.818+0000 7fe131540640 1 -- 172.21.15.50:0/1811438695 wait complete. 2023-10-19T10:37:44.831 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:44.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.938+0000 7f2030303640 1 Processor -- start 2023-10-19T10:37:44.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.939+0000 7f2030303640 1 -- start start 2023-10-19T10:37:44.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.939+0000 7f2030303640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f20280b26e0 0x7f202806f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.939+0000 7f2030303640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2028145880 0x7f20280ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.939+0000 7f2030303640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280701b0 con 0x7f20280ce560 2023-10-19T10:37:44.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.939+0000 7f2030303640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280ce9f0 con 0x7f20280b26e0 2023-10-19T10:37:44.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.939+0000 7f2030303640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202806f910 con 0x7f2028145880 2023-10-19T10:37:44.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e09f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f20280b26e0 0x7f202806f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e8a0640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f20280ce560 0x7f20281600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40728/0 (socket says 172.21.15.50:40728) 2023-10-19T10:37:44.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e8a0640 1 -- 172.21.15.50:0/1868457045 learned_addr learned my addr 172.21.15.50:0/1868457045 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:44.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202d89e640 1 --2- 172.21.15.50:0/1868457045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2028145880 0x7f20280ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e09f640 1 -- 172.21.15.50:0/1868457045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2028145880 msgr2=0x7f20280ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:44.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e09f640 1 --2- 172.21.15.50:0/1868457045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2028145880 0x7f20280ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e09f640 1 -- 172.21.15.50:0/1868457045 >> v1:172.21.15.50:6789/0 conn(0x7f20280ce560 legacy=0x7f20281600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202d09d640 1 -- 172.21.15.50:0/1868457045 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566890138 0 0) 0x7f20280701b0 con 0x7f20280ce560 2023-10-19T10:37:44.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.940+0000 7f202e09f640 1 -- 172.21.15.50:0/1868457045 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202806fbf0 con 0x7f20280b26e0 2023-10-19T10:37:44.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f202e09f640 1 --2- 172.21.15.50:0/1868457045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280b26e0 0x7f202806f2a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f20240097a0 tx=0x7f202402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55cd8b56fbf52c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f202d09d640 1 -- 172.21.15.50:0/1868457045 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202402e070 con 0x7f20280b26e0 2023-10-19T10:37:44.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f202d09d640 1 -- 172.21.15.50:0/1868457045 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20240315a0 con 0x7f20280b26e0 2023-10-19T10:37:44.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f202d09d640 1 -- 172.21.15.50:0/1868457045 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024031930 con 0x7f20280b26e0 2023-10-19T10:37:44.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f2030303640 1 -- 172.21.15.50:0/1868457045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280b26e0 msgr2=0x7f202806f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f2030303640 1 --2- 172.21.15.50:0/1868457045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280b26e0 0x7f202806f2a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f20240097a0 tx=0x7f202402b840 comp rx=0 tx=0).stop 2023-10-19T10:37:44.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f2030303640 1 -- 172.21.15.50:0/1868457045 shutdown_connections 2023-10-19T10:37:44.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f2030303640 1 --2- 172.21.15.50:0/1868457045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2028145880 0x7f20280ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.941+0000 7f2030303640 1 --2- 172.21.15.50:0/1868457045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280b26e0 0x7f202806f2a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.942+0000 7f2030303640 1 -- 172.21.15.50:0/1868457045 >> 172.21.15.50:0/1868457045 conn(0x7f20280ccf60 msgr2=0x7f20280cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:44.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.942+0000 7f2030303640 1 -- 172.21.15.50:0/1868457045 shutdown_connections 2023-10-19T10:37:44.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.942+0000 7f2030303640 1 -- 172.21.15.50:0/1868457045 wait complete. 2023-10-19T10:37:44.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 Processor -- start 2023-10-19T10:37:44.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 -- start start 2023-10-19T10:37:44.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20280b26e0 0x7f20280aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 0x7f20280acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2028145880 0x7f20280c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280ad4a0 con 0x7f20280ce560 2023-10-19T10:37:44.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202814ea50 con 0x7f2028145880 2023-10-19T10:37:44.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f2030303640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280b3100 con 0x7f20280b26e0 2023-10-19T10:37:44.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.943+0000 7f202d89e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 0x7f20280acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 0x7f20280acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57042/0 (socket says 172.21.15.50:57042) 2023-10-19T10:37:44.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 -- 172.21.15.50:0/441452729 learned_addr learned my addr 172.21.15.50:0/441452729 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:44.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202e8a0640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2028145880 0x7f20280c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202e09f640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20280b26e0 0x7f20280aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 -- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20280b26e0 msgr2=0x7f20280aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20280b26e0 0x7f20280aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 -- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2028145880 msgr2=0x7f20280c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:44.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2028145880 0x7f20280c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:44.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 -- 172.21.15.50:0/441452729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024009400 con 0x7f20280ce560 2023-10-19T10:37:44.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202e8a0640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2028145880 0x7f20280c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:44.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202e09f640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20280b26e0 0x7f20280aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:44.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f202d89e640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 0x7f20280acf60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f201800c7f0 tx=0x7f201800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018004220 con 0x7f20280ce560 2023-10-19T10:37:44.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20180043a0 con 0x7f20280ce560 2023-10-19T10:37:44.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018007860 con 0x7f20280ce560 2023-10-19T10:37:44.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280ad700 con 0x7f20280ce560 2023-10-19T10:37:44.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.944+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280adcc0 con 0x7f20280ce560 2023-10-19T10:37:44.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.945+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202804e420 con 0x7f20280ce560 2023-10-19T10:37:44.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.946+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2018007bd0 con 0x7f20280ce560 2023-10-19T10:37:44.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.946+0000 7f201effd640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f200c06a050 0x7f200c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:44.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.946+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2018088940 con 0x7f20280ce560 2023-10-19T10:37:44.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.946+0000 7f202e09f640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f200c06a050 0x7f200c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:44.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.947+0000 7f202e09f640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f200c06a050 0x7f200c06c510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2024036000 tx=0x7f2024008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:44.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:44.950+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2018053c60 con 0x7f20280ce560 2023-10-19T10:37:45.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.124+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f20280afbe0 con 0x7f20280ce560 2023-10-19T10:37:45.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.124+0000 7f201effd640 1 -- 172.21.15.50:0/441452729 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f201801e020 con 0x7f20280ce560 2023-10-19T10:37:45.127 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:45.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f200c06a050 msgr2=0x7f200c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f200c06a050 0x7f200c06c510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2024036000 tx=0x7f2024008030 comp rx=0 tx=0).stop 2023-10-19T10:37:45.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 msgr2=0x7f20280acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 0x7f20280acf60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f201800c7f0 tx=0x7f201800ccb0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 shutdown_connections 2023-10-19T10:37:45.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f200c06a050 0x7f200c06c510 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2028145880 0x7f20280c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20280ce560 0x7f20280acf60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 --2- 172.21.15.50:0/441452729 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20280b26e0 0x7f20280aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 >> 172.21.15.50:0/441452729 conn(0x7f20280ccf60 msgr2=0x7f20280b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:45.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 shutdown_connections 2023-10-19T10:37:45.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.126+0000 7f2030303640 1 -- 172.21.15.50:0/441452729 wait complete. 2023-10-19T10:37:45.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:45.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.248+0000 7f25aa9ab640 1 Processor -- start 2023-10-19T10:37:45.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.249+0000 7f25aa9ab640 1 -- start start 2023-10-19T10:37:45.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.249+0000 7f25aa9ab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 0x7f25a40ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.249+0000 7f25aa9ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 0x7f25a415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.249+0000 7f25aa9ab640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a4158310 con 0x7f25a40d0b20 2023-10-19T10:37:45.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.249+0000 7f25aa9ab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a415b910 con 0x7f25a40d0f00 2023-10-19T10:37:45.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.249+0000 7f25aa9ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40cc660 con 0x7f25a40ce570 2023-10-19T10:37:45.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a37fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 0x7f25a40ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a8f48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 0x7f25a415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a3fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f25a40d0b20 0x7f25a40cde60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:35510/0 (socket says 172.21.15.50:35510) 2023-10-19T10:37:45.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a37fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 0x7f25a40ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:44672/0 (socket says 172.21.15.50:44672) 2023-10-19T10:37:45.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a8f48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 0x7f25a415b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:45064/0 (socket says 172.21.15.50:45064) 2023-10-19T10:37:45.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a3fff640 1 -- 172.21.15.50:0/2873018224 learned_addr learned my addr 172.21.15.50:0/2873018224 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:45.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a8f48640 1 -- 172.21.15.50:0/2873018224 >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 msgr2=0x7f25a40ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:45.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a8f48640 1 --2- 172.21.15.50:0/2873018224 >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 0x7f25a40ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a8f48640 1 -- 172.21.15.50:0/2873018224 >> v1:172.21.15.183:6789/0 conn(0x7f25a40d0b20 legacy=0x7f25a40cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a8f48640 1 -- 172.21.15.50:0/2873018224 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40cc940 con 0x7f25a40ce570 2023-10-19T10:37:45.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.250+0000 7f25a37fe640 1 --2- 172.21.15.50:0/2873018224 >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 0x7f25a40ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:45.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.251+0000 7f25a8f48640 1 --2- 172.21.15.50:0/2873018224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 0x7f25a415b3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d90 tx=0x7f259c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89baeacc7630dd6f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.251+0000 7f25a2ffd640 1 -- 172.21.15.50:0/2873018224 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c02cbc0 con 0x7f25a40ce570 2023-10-19T10:37:45.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.251+0000 7f25a2ffd640 1 -- 172.21.15.50:0/2873018224 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c02cd20 con 0x7f25a40ce570 2023-10-19T10:37:45.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.251+0000 7f25a2ffd640 1 -- 172.21.15.50:0/2873018224 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c034660 con 0x7f25a40ce570 2023-10-19T10:37:45.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 -- 172.21.15.50:0/2873018224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 msgr2=0x7f25a415b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/2873018224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 0x7f25a415b3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d90 tx=0x7f259c02be10 comp rx=0 tx=0).stop 2023-10-19T10:37:45.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 -- 172.21.15.50:0/2873018224 shutdown_connections 2023-10-19T10:37:45.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/2873018224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40ce570 0x7f25a415b3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/2873018224 >> v2:172.21.15.183:3300/0 conn(0x7f25a40d0f00 0x7f25a40ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 -- 172.21.15.50:0/2873018224 >> 172.21.15.50:0/2873018224 conn(0x7f25a406f3f0 msgr2=0x7f25a406f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:45.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 -- 172.21.15.50:0/2873018224 shutdown_connections 2023-10-19T10:37:45.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.252+0000 7f25aa9ab640 1 -- 172.21.15.50:0/2873018224 wait complete. 2023-10-19T10:37:45.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.253+0000 7f25aa9ab640 1 Processor -- start 2023-10-19T10:37:45.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.253+0000 7f25aa9ab640 1 -- start start 2023-10-19T10:37:45.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25aa9ab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 0x7f25a41579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25aa9ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40d0b20 0x7f25a40ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25aa9ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25a40ac2c0 0x7f25a40c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25aa9ab640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40c3dc0 con 0x7f25a40ce570 2023-10-19T10:37:45.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25aa9ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40ad7f0 con 0x7f25a40ac2c0 2023-10-19T10:37:45.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25aa9ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40ac860 con 0x7f25a40d0b20 2023-10-19T10:37:45.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 0x7f25a41579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 0x7f25a41579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57050/0 (socket says 172.21.15.50:57050) 2023-10-19T10:37:45.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 -- 172.21.15.50:0/3234246967 learned_addr learned my addr 172.21.15.50:0/3234246967 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:45.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a37fe640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40d0b20 0x7f25a40ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a8f48640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25a40ac2c0 0x7f25a40c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 -- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40d0b20 msgr2=0x7f25a40ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40d0b20 0x7f25a40ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 -- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25a40ac2c0 msgr2=0x7f25a40c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25a40ac2c0 0x7f25a40c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a3fff640 1 -- 172.21.15.50:0/3234246967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2594009400 con 0x7f25a40ce570 2023-10-19T10:37:45.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.254+0000 7f25a8f48640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25a40ac2c0 0x7f25a40c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:45.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.255+0000 7f25a3fff640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 0x7f25a41579c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f259800c4a0 tx=0x7f259800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.255+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598005ce0 con 0x7f25a40ce570 2023-10-19T10:37:45.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.255+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598005e40 con 0x7f25a40ce570 2023-10-19T10:37:45.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.255+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c009a70 con 0x7f25a40ce570 2023-10-19T10:37:45.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.255+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598007270 con 0x7f25a40ce570 2023-10-19T10:37:45.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.255+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a40ace20 con 0x7f25a40ce570 2023-10-19T10:37:45.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.256+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a40c3f30 con 0x7f25a40ce570 2023-10-19T10:37:45.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.256+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2598007490 con 0x7f25a40ce570 2023-10-19T10:37:45.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.257+0000 7f25a0ff9640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f259006a050 0x7f259006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.257+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2598087d40 con 0x7f25a40ce570 2023-10-19T10:37:45.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.257+0000 7f25a37fe640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f259006a050 0x7f259006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.258+0000 7f25a37fe640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f259006a050 0x7f259006c510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25a406c810 tx=0x7f2594008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.262+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f25a40c3f30 con 0x7f25a40ce570 2023-10-19T10:37:45.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.449+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25a40c3f00 con 0x7f25a40ce570 2023-10-19T10:37:45.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.450+0000 7f25a0ff9640 1 -- 172.21.15.50:0/3234246967 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f2598017090 con 0x7f25a40ce570 2023-10-19T10:37:45.453 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:45.454 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:45.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f259006a050 msgr2=0x7f259006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f259006a050 0x7f259006c510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25a406c810 tx=0x7f2594008040 comp rx=0 tx=0).stop 2023-10-19T10:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 msgr2=0x7f25a41579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 0x7f25a41579c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f259800c4a0 tx=0x7f259800c960 comp rx=0 tx=0).stop 2023-10-19T10:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 shutdown_connections 2023-10-19T10:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f259006a050 0x7f259006c510 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25a40ac2c0 0x7f25a40c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f25a40d0b20 0x7f25a40ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 --2- 172.21.15.50:0/3234246967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25a40ce570 0x7f25a41579c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 >> 172.21.15.50:0/3234246967 conn(0x7f25a406f3f0 msgr2=0x7f25a406b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:45.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 shutdown_connections 2023-10-19T10:37:45.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.452+0000 7f25aa9ab640 1 -- 172.21.15.50:0/3234246967 wait complete. 2023-10-19T10:37:45.466 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:45.467 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:45.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.574+0000 7f60a3671640 1 Processor -- start 2023-10-19T10:37:45.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.574+0000 7f60a3671640 1 -- start start 2023-10-19T10:37:45.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a3671640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f609c15c660 0x7f609c06a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a3671640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 0x7f609c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a3671640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609c070780 con 0x7f609c160240 2023-10-19T10:37:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a3671640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c0cca10 con 0x7f609c15c660 2023-10-19T10:37:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a3671640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c06a750 con 0x7f609c0cd460 2023-10-19T10:37:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a0c0c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f609c160240 0x7f609c0ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40734/0 (socket says 172.21.15.50:40734) 2023-10-19T10:37:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.575+0000 7f60a0c0c640 1 -- 172.21.15.50:0/3251160319 learned_addr learned my addr 172.21.15.50:0/3251160319 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a140d640 1 --2- 172.21.15.50:0/3251160319 >> v2:172.21.15.183:3300/0 conn(0x7f609c15c660 0x7f609c06a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a1c0e640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 0x7f609c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935298723 0 0) 0x7f609c070780 con 0x7f609c160240 2023-10-19T10:37:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6084003520 con 0x7f609c160240 2023-10-19T10:37:45.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292687510 0 0) 0x7f6084003520 con 0x7f609c160240 2023-10-19T10:37:45.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 msgr2=0x7f609c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:45.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 0x7f609c070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 >> v2:172.21.15.183:3300/0 conn(0x7f609c15c660 msgr2=0x7f609c06a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:45.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 --2- 172.21.15.50:0/3251160319 >> v2:172.21.15.183:3300/0 conn(0x7f609c15c660 0x7f609c06a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c160720 con 0x7f609c160240 2023-10-19T10:37:45.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f6090001670 con 0x7f609c160240 2023-10-19T10:37:45.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a140d640 1 --2- 172.21.15.50:0/3251160319 >> v2:172.21.15.183:3300/0 conn(0x7f609c15c660 0x7f609c06a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:45.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 >> v1:172.21.15.50:6789/0 conn(0x7f609c160240 legacy=0x7f609c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6084003970 0x7f6084003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60840047c0 0x7f6084004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 0x7f6084005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0cca10 con 0x7f60840047c0 2023-10-19T10:37:45.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c06a750 con 0x7f6084003970 2023-10-19T10:37:45.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608400b980 con 0x7f609c0cd460 2023-10-19T10:37:45.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60840047c0 0x7f6084004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a1c0e640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 0x7f6084005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a140d640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6084003970 0x7f6084003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a0c0c640 1 -- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 msgr2=0x7f6084005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:45.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f609c0cd460 0x7f6084005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a0c0c640 1 -- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6084003970 msgr2=0x7f6084003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:45.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6084003970 0x7f6084003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.576+0000 7f60a0c0c640 1 -- 172.21.15.50:0/3251160319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084003520 con 0x7f60840047c0 2023-10-19T10:37:45.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.577+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60840047c0 0x7f6084004c40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f609000c290 tx=0x7f60840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8185dc68707d3e76 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.577+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090005ad0 con 0x7f60840047c0 2023-10-19T10:37:45.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.577+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090005c30 con 0x7f60840047c0 2023-10-19T10:37:45.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.577+0000 7f609bfff640 1 -- 172.21.15.50:0/3251160319 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609002c8d0 con 0x7f60840047c0 2023-10-19T10:37:45.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a3671640 1 -- 172.21.15.50:0/3251160319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60840047c0 msgr2=0x7f6084004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a3671640 1 --2- 172.21.15.50:0/3251160319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60840047c0 0x7f6084004c40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f609000c290 tx=0x7f60840057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a1c0e640 1 -- 172.21.15.50:0/3251160319 reap_dead start 2023-10-19T10:37:45.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a3671640 1 -- 172.21.15.50:0/3251160319 shutdown_connections 2023-10-19T10:37:45.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a3671640 1 -- 172.21.15.50:0/3251160319 >> 172.21.15.50:0/3251160319 conn(0x7f609c0cbc00 msgr2=0x7f609c0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:45.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a3671640 1 -- 172.21.15.50:0/3251160319 shutdown_connections 2023-10-19T10:37:45.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.578+0000 7f60a3671640 1 -- 172.21.15.50:0/3251160319 wait complete. 2023-10-19T10:37:45.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.579+0000 7f60a3671640 1 Processor -- start 2023-10-19T10:37:45.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.579+0000 7f60a3671640 1 -- start start 2023-10-19T10:37:45.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a3671640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60840047c0 0x7f609c1f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a3671640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 0x7f609c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a3671640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c15a580 0x7f609c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a3671640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1f3300 con 0x7f609c0b4550 2023-10-19T10:37:45.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a3671640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c15af40 con 0x7f609c15a580 2023-10-19T10:37:45.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a3671640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1f2eb0 con 0x7f60840047c0 2023-10-19T10:37:45.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 0x7f609c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 0x7f609c0b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57066/0 (socket says 172.21.15.50:57066) 2023-10-19T10:37:45.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 -- 172.21.15.50:0/2793574355 learned_addr learned my addr 172.21.15.50:0/2793574355 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:45.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a1c0e640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c15a580 0x7f609c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a140d640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60840047c0 0x7f609c1f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 -- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60840047c0 msgr2=0x7f609c1f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60840047c0 0x7f609c1f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 -- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c15a580 msgr2=0x7f609c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c15a580 0x7f609c15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a0c0c640 1 -- 172.21.15.50:0/2793574355 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c1f23b0 con 0x7f609c0b4550 2023-10-19T10:37:45.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.580+0000 7f60a1c0e640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c15a580 0x7f609c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:45.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.581+0000 7f60a0c0c640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 0x7f609c0b49d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f609002c470 tx=0x7f6090000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.581+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609003c040 con 0x7f609c0b4550 2023-10-19T10:37:45.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.581+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609002ddb0 con 0x7f609c0b4550 2023-10-19T10:37:45.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.581+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c0b20f0 con 0x7f609c0b4550 2023-10-19T10:37:45.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.581+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60900072b0 con 0x7f609c0b4550 2023-10-19T10:37:45.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.581+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c1f28f0 con 0x7f609c0b4550 2023-10-19T10:37:45.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.582+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c1f3060 con 0x7f609c0b4550 2023-10-19T10:37:45.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.582+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6090032070 con 0x7f609c0b4550 2023-10-19T10:37:45.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.583+0000 7f6099ffb640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f608806a080 0x7f608806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.583+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f60900a9aa0 con 0x7f609c0b4550 2023-10-19T10:37:45.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.583+0000 7f60a140d640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f608806a080 0x7f608806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.584+0000 7f60a140d640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f608806a080 0x7f608806c540 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f609c0c2660 tx=0x7f608c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.587+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f609c1f3060 con 0x7f609c0b4550 2023-10-19T10:37:45.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.761+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f609c1f3060 con 0x7f609c0b4550 2023-10-19T10:37:45.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.761+0000 7f6099ffb640 1 -- 172.21.15.50:0/2793574355 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f609c1f3060 con 0x7f609c0b4550 2023-10-19T10:37:45.765 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:45.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f608806a080 msgr2=0x7f608806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f608806a080 0x7f608806c540 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f609c0c2660 tx=0x7f608c00b040 comp rx=0 tx=0).stop 2023-10-19T10:37:45.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 msgr2=0x7f609c0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 0x7f609c0b49d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f609002c470 tx=0x7f6090000fe0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 shutdown_connections 2023-10-19T10:37:45.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f608806a080 0x7f608806c540 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c15a580 0x7f609c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f609c0b4550 0x7f609c0b49d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 --2- 172.21.15.50:0/2793574355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60840047c0 0x7f609c1f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 >> 172.21.15.50:0/2793574355 conn(0x7f609c0cbc00 msgr2=0x7f609c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:45.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 shutdown_connections 2023-10-19T10:37:45.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.764+0000 7f60a3671640 1 -- 172.21.15.50:0/2793574355 wait complete. 2023-10-19T10:37:45.778 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.886+0000 7fb217911640 1 Processor -- start 2023-10-19T10:37:45.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb217911640 1 -- start start 2023-10-19T10:37:45.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb217911640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb210145640 0x7fb2100b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb217911640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb21015d250 0x7fb2100b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb217911640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21015d6a0 con 0x7fb210160d70 2023-10-19T10:37:45.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb217911640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21015d8e0 con 0x7fb210145640 2023-10-19T10:37:45.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb217911640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb210160290 con 0x7fb21015d250 2023-10-19T10:37:45.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb2156ad640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb210145640 0x7fb2100b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb214eac640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb210160d70 0x7fb21015cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40744/0 (socket says 172.21.15.50:40744) 2023-10-19T10:37:45.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.887+0000 7fb214eac640 1 -- 172.21.15.50:0/1407613051 learned_addr learned my addr 172.21.15.50:0/1407613051 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:45.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb2156ad640 1 -- 172.21.15.50:0/1407613051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb21015d250 msgr2=0x7fb2100b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:45.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb215eae640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb21015d250 0x7fb2100b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb207fff640 1 -- 172.21.15.50:0/1407613051 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090552709 0 0) 0x7fb200001110 con 0x7fb210160d70 2023-10-19T10:37:45.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb2156ad640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb21015d250 0x7fb2100b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb2156ad640 1 -- 172.21.15.50:0/1407613051 >> v1:172.21.15.50:6789/0 conn(0x7fb210160d70 legacy=0x7fb21015cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb2156ad640 1 -- 172.21.15.50:0/1407613051 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210160570 con 0x7fb210145640 2023-10-19T10:37:45.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.888+0000 7fb215eae640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb21015d250 0x7fb2100b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:45.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb2156ad640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100b1d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0097a0 tx=0x7fb20c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9470fcc73a058bfa server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb207fff640 1 -- 172.21.15.50:0/1407613051 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c031830 con 0x7fb210145640 2023-10-19T10:37:45.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb207fff640 1 -- 172.21.15.50:0/1407613051 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c031990 con 0x7fb210145640 2023-10-19T10:37:45.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb207fff640 1 -- 172.21.15.50:0/1407613051 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c031d20 con 0x7fb210145640 2023-10-19T10:37:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 -- 172.21.15.50:0/1407613051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 msgr2=0x7fb2100b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100b1d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0097a0 tx=0x7fb20c02b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 -- 172.21.15.50:0/1407613051 shutdown_connections 2023-10-19T10:37:45.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb21015d250 0x7fb2100b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 --2- 172.21.15.50:0/1407613051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100b1d10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 -- 172.21.15.50:0/1407613051 >> 172.21.15.50:0/1407613051 conn(0x7fb2100ccd10 msgr2=0x7fb2100cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:45.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 -- 172.21.15.50:0/1407613051 shutdown_connections 2023-10-19T10:37:45.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.889+0000 7fb217911640 1 -- 172.21.15.50:0/1407613051 wait complete. 2023-10-19T10:37:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.890+0000 7fb217911640 1 Processor -- start 2023-10-19T10:37:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 -- start start 2023-10-19T10:37:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb21015d250 0x7fb2100c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210160d70 0x7fb2100c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100c3370 con 0x7fb210145640 2023-10-19T10:37:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100c8220 con 0x7fb21015d250 2023-10-19T10:37:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb217911640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100c8500 con 0x7fb210160d70 2023-10-19T10:37:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb2156ad640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb2156ad640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57086/0 (socket says 172.21.15.50:57086) 2023-10-19T10:37:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.891+0000 7fb2156ad640 1 -- 172.21.15.50:0/3516615041 learned_addr learned my addr 172.21.15.50:0/3516615041 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb2156ad640 1 -- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210160d70 msgr2=0x7fb2100c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb2156ad640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210160d70 0x7fb2100c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb2156ad640 1 -- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb21015d250 msgr2=0x7fb2100c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb2156ad640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb21015d250 0x7fb2100c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb2156ad640 1 -- 172.21.15.50:0/3516615041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c009400 con 0x7fb210145640 2023-10-19T10:37:45.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb2156ad640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100c2e30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0097a0 tx=0x7fb20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c004230 con 0x7fb210145640 2023-10-19T10:37:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c004390 con 0x7fb210145640 2023-10-19T10:37:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2100c87e0 con 0x7fb210145640 2023-10-19T10:37:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c03a930 con 0x7fb210145640 2023-10-19T10:37:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.892+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2100c8da0 con 0x7fb210145640 2023-10-19T10:37:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.893+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21004e420 con 0x7fb210145640 2023-10-19T10:37:45.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.894+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb20c03aa90 con 0x7fb210145640 2023-10-19T10:37:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.894+0000 7fb205ffb640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1f406a050 0x7fb1f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.894+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb20c0aabd0 con 0x7fb210145640 2023-10-19T10:37:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.894+0000 7fb214eac640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1f406a050 0x7fb1f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.895+0000 7fb214eac640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1f406a050 0x7fb1f406c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb200007fd0 tx=0x7fb200001300 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:45.899+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb20c075f90 con 0x7fb210145640 2023-10-19T10:37:46.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.073+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb21015fa60 con 0x7fb210145640 2023-10-19T10:37:46.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.073+0000 7fb205ffb640 1 -- 172.21.15.50:0/3516615041 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb20c0350a0 con 0x7fb210145640 2023-10-19T10:37:46.076 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:46.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1f406a050 msgr2=0x7fb1f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1f406a050 0x7fb1f406c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb200007fd0 tx=0x7fb200001300 comp rx=0 tx=0).stop 2023-10-19T10:37:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 msgr2=0x7fb2100c2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100c2e30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0097a0 tx=0x7fb20c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 shutdown_connections 2023-10-19T10:37:46.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1f406a050 0x7fb1f406c510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210160d70 0x7fb2100c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb21015d250 0x7fb2100c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.075+0000 7fb217911640 1 --2- 172.21.15.50:0/3516615041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210145640 0x7fb2100c2e30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.076+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 >> 172.21.15.50:0/3516615041 conn(0x7fb2100ccd10 msgr2=0x7fb21006a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:46.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.076+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 shutdown_connections 2023-10-19T10:37:46.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.076+0000 7fb217911640 1 -- 172.21.15.50:0/3516615041 wait complete. 2023-10-19T10:37:46.089 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:46.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.196+0000 7f277f673640 1 Processor -- start 2023-10-19T10:37:46.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.196+0000 7f277f673640 1 -- start start 2023-10-19T10:37:46.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277f673640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f27780b4070 0x7f27780b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277f673640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f277815cdf0 0x7f27780cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277f673640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27780d3480 con 0x7f277815c960 2023-10-19T10:37:46.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277f673640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27780b4990 con 0x7f277815cdf0 2023-10-19T10:37:46.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277f673640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277815d280 con 0x7f27780b4070 2023-10-19T10:37:46.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f277815cdf0 0x7f27780cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f277815cdf0 0x7f27780cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57096/0 (socket says 172.21.15.50:57096) 2023-10-19T10:37:46.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 -- 172.21.15.50:0/1575937 learned_addr learned my addr 172.21.15.50:0/1575937 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:46.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277d40f640 1 --2- 172.21.15.50:0/1575937 >> v2:172.21.15.183:3300/0 conn(0x7f27780b4070 0x7f27780b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 -- 172.21.15.50:0/1575937 >> v2:172.21.15.183:3300/0 conn(0x7f27780b4070 msgr2=0x7f27780b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 --2- 172.21.15.50:0/1575937 >> v2:172.21.15.183:3300/0 conn(0x7f27780b4070 0x7f27780b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 -- 172.21.15.50:0/1575937 >> v1:172.21.15.183:6789/0 conn(0x7f277815c960 legacy=0x7f27780d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277dc10640 1 -- 172.21.15.50:0/1575937 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277815d560 con 0x7f277815cdf0 2023-10-19T10:37:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.197+0000 7f277d40f640 1 --2- 172.21.15.50:0/1575937 >> v2:172.21.15.183:3300/0 conn(0x7f27780b4070 0x7f27780b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.198+0000 7f277dc10640 1 --2- 172.21.15.50:0/1575937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f277815cdf0 0x7f27780cd4b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f277808a3b0 tx=0x7f277002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ace226c3b35e333 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.198+0000 7f276ffff640 1 -- 172.21.15.50:0/1575937 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277002ca80 con 0x7f277815cdf0 2023-10-19T10:37:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.198+0000 7f276ffff640 1 -- 172.21.15.50:0/1575937 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277002cbe0 con 0x7f277815cdf0 2023-10-19T10:37:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.198+0000 7f276ffff640 1 -- 172.21.15.50:0/1575937 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700343f0 con 0x7f277815cdf0 2023-10-19T10:37:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 -- 172.21.15.50:0/1575937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f277815cdf0 msgr2=0x7f27780cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 --2- 172.21.15.50:0/1575937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f277815cdf0 0x7f27780cd4b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f277808a3b0 tx=0x7f277002bee0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 -- 172.21.15.50:0/1575937 shutdown_connections 2023-10-19T10:37:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 --2- 172.21.15.50:0/1575937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f277815cdf0 0x7f27780cd4b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 --2- 172.21.15.50:0/1575937 >> v2:172.21.15.183:3300/0 conn(0x7f27780b4070 0x7f27780b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 -- 172.21.15.50:0/1575937 >> 172.21.15.50:0/1575937 conn(0x7f277809b970 msgr2=0x7f27780d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:46.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 -- 172.21.15.50:0/1575937 shutdown_connections 2023-10-19T10:37:46.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.199+0000 7f277f673640 1 -- 172.21.15.50:0/1575937 wait complete. 2023-10-19T10:37:46.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.200+0000 7f277f673640 1 Processor -- start 2023-10-19T10:37:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.200+0000 7f277f673640 1 -- start start 2023-10-19T10:37:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277f673640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 0x7f2778157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277f673640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f277815c960 0x7f27780cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277f673640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27780cc6c0 0x7f27781f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277f673640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780c7520 con 0x7f27780b4070 2023-10-19T10:37:46.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277f673640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780c7770 con 0x7f27780cc6c0 2023-10-19T10:37:46.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277f673640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27780c7a50 con 0x7f277815c960 2023-10-19T10:37:46.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 0x7f2778157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 0x7f2778157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57108/0 (socket says 172.21.15.50:57108) 2023-10-19T10:37:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 -- 172.21.15.50:0/3498334096 learned_addr learned my addr 172.21.15.50:0/3498334096 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277dc10640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27780cc6c0 0x7f27781f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277cc0e640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f277815c960 0x7f27780cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 -- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f277815c960 msgr2=0x7f27780cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f277815c960 0x7f27780cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 -- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27780cc6c0 msgr2=0x7f27781f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27780cc6c0 0x7f27781f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 -- 172.21.15.50:0/3498334096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770009b30 con 0x7f27780b4070 2023-10-19T10:37:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277dc10640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27780cc6c0 0x7f27781f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f277d40f640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 0x7f2778157650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f277400c7f0 tx=0x7f277400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.201+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774005b90 con 0x7f27780b4070 2023-10-19T10:37:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.202+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2774005cf0 con 0x7f27780b4070 2023-10-19T10:37:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.202+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780c7d30 con 0x7f27780b4070 2023-10-19T10:37:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.202+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774007a30 con 0x7f27780b4070 2023-10-19T10:37:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.202+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27780c82f0 con 0x7f27780b4070 2023-10-19T10:37:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.203+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277804e420 con 0x7f27780b4070 2023-10-19T10:37:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.203+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2774007b90 con 0x7f27780b4070 2023-10-19T10:37:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.203+0000 7f276dffb640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f275c06a050 0x7f275c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.204+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2774088860 con 0x7f27780b4070 2023-10-19T10:37:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.204+0000 7f277cc0e640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f275c06a050 0x7f275c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.204+0000 7f277cc0e640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f275c06a050 0x7f275c06c510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2768007fd0 tx=0x7f2768009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.208+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f27740576b0 con 0x7f27780b4070 2023-10-19T10:37:46.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.383+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27780b4ce0 con 0x7f27780b4070 2023-10-19T10:37:46.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.383+0000 7f276dffb640 1 -- 172.21.15.50:0/3498334096 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2774057240 con 0x7f27780b4070 2023-10-19T10:37:46.386 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:46.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f275c06a050 msgr2=0x7f275c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f275c06a050 0x7f275c06c510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2768007fd0 tx=0x7f2768009040 comp rx=0 tx=0).stop 2023-10-19T10:37:46.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 msgr2=0x7f2778157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 0x7f2778157650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f277400c7f0 tx=0x7f277400cbf0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 shutdown_connections 2023-10-19T10:37:46.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f275c06a050 0x7f275c06c510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27780cc6c0 0x7f27781f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f277815c960 0x7f27780cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 --2- 172.21.15.50:0/3498334096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27780b4070 0x7f2778157650 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 >> 172.21.15.50:0/3498334096 conn(0x7f277809b970 msgr2=0x7f27780b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:46.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 shutdown_connections 2023-10-19T10:37:46.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.386+0000 7f277f673640 1 -- 172.21.15.50:0/3498334096 wait complete. 2023-10-19T10:37:46.399 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.506+0000 7f27b0c46640 1 Processor -- start 2023-10-19T10:37:46.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.507+0000 7f27b0c46640 1 -- start start 2023-10-19T10:37:46.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.507+0000 7f27b0c46640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f27ac0b38f0 0x7f27ac0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.507+0000 7f27b0c46640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 0x7f27ac0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.507+0000 7f27b0c46640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac0af250 con 0x7f27ac0b3090 2023-10-19T10:37:46.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.507+0000 7f27b0c46640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0b0190 con 0x7f27ac0b38f0 2023-10-19T10:37:46.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.507+0000 7f27b0c46640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0af510 con 0x7f27ac161210 2023-10-19T10:37:46.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aa59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f27ac0b38f0 0x7f27ac0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 0x7f27ac0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 0x7f27ac0b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:45130/0 (socket says 172.21.15.50:45130) 2023-10-19T10:37:46.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 -- 172.21.15.50:0/775832554 learned_addr learned my addr 172.21.15.50:0/775832554 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:46.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27a9d9b640 1 -- 172.21.15.50:0/775832554 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049107389 0 0) 0x7f27ac0af250 con 0x7f27ac0b3090 2023-10-19T10:37:46.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27a9d9b640 1 -- 172.21.15.50:0/775832554 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2798003520 con 0x7f27ac0b3090 2023-10-19T10:37:46.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 -- 172.21.15.50:0/775832554 >> v2:172.21.15.183:3300/0 conn(0x7f27ac0b38f0 msgr2=0x7f27ac0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:46.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 --2- 172.21.15.50:0/775832554 >> v2:172.21.15.183:3300/0 conn(0x7f27ac0b38f0 0x7f27ac0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 -- 172.21.15.50:0/775832554 >> v1:172.21.15.183:6789/0 conn(0x7f27ac0b3090 legacy=0x7f27ac160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aad9d640 1 -- 172.21.15.50:0/775832554 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0af7f0 con 0x7f27ac161210 2023-10-19T10:37:46.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.508+0000 7f27aa59c640 1 --2- 172.21.15.50:0/775832554 >> v2:172.21.15.183:3300/0 conn(0x7f27ac0b38f0 0x7f27ac0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:46.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.509+0000 7f27aad9d640 1 --2- 172.21.15.50:0/775832554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 0x7f27ac0b05b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009e50 tx=0x7f27a402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a669a94fac31575e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.509+0000 7f27a9d9b640 1 -- 172.21.15.50:0/775832554 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a402cb00 con 0x7f27ac161210 2023-10-19T10:37:46.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.509+0000 7f27a9d9b640 1 -- 172.21.15.50:0/775832554 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a402cc60 con 0x7f27ac161210 2023-10-19T10:37:46.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.509+0000 7f27a9d9b640 1 -- 172.21.15.50:0/775832554 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a40343f0 con 0x7f27ac161210 2023-10-19T10:37:46.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 -- 172.21.15.50:0/775832554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 msgr2=0x7f27ac0b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 --2- 172.21.15.50:0/775832554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 0x7f27ac0b05b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009e50 tx=0x7f27a402bf60 comp rx=0 tx=0).stop 2023-10-19T10:37:46.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 -- 172.21.15.50:0/775832554 shutdown_connections 2023-10-19T10:37:46.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 --2- 172.21.15.50:0/775832554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac161210 0x7f27ac0b05b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 --2- 172.21.15.50:0/775832554 >> v2:172.21.15.183:3300/0 conn(0x7f27ac0b38f0 0x7f27ac0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 -- 172.21.15.50:0/775832554 >> 172.21.15.50:0/775832554 conn(0x7f27ac09b960 msgr2=0x7f27ac0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:46.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 -- 172.21.15.50:0/775832554 shutdown_connections 2023-10-19T10:37:46.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.510+0000 7f27b0c46640 1 -- 172.21.15.50:0/775832554 wait complete. 2023-10-19T10:37:46.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.511+0000 7f27b0c46640 1 Processor -- start 2023-10-19T10:37:46.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.511+0000 7f27b0c46640 1 -- start start 2023-10-19T10:37:46.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27b0c46640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 0x7f27ac156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27b0c46640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac0b38f0 0x7f27ac0b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27b0c46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27ac161210 0x7f27ac0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27b0c46640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0c7150 con 0x7f27ac0b3090 2023-10-19T10:37:46.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27b0c46640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0cc290 con 0x7f27ac161210 2023-10-19T10:37:46.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27b0c46640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0b52c0 con 0x7f27ac0b38f0 2023-10-19T10:37:46.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 0x7f27ac156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 0x7f27ac156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57122/0 (socket says 172.21.15.50:57122) 2023-10-19T10:37:46.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 -- 172.21.15.50:0/3533315542 learned_addr learned my addr 172.21.15.50:0/3533315542 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:46.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aad9d640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27ac161210 0x7f27ac0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27a3fff640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac0b38f0 0x7f27ac0b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 -- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac0b38f0 msgr2=0x7f27ac0b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac0b38f0 0x7f27ac0b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 -- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27ac161210 msgr2=0x7f27ac0b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27ac161210 0x7f27ac0b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.512+0000 7f27aa59c640 1 -- 172.21.15.50:0/3533315542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0093a0 con 0x7f27ac0b3090 2023-10-19T10:37:46.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.513+0000 7f27aa59c640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 0x7f27ac156af0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f279c00ebe0 tx=0x7f279c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.513+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c009750 con 0x7f27ac0b3090 2023-10-19T10:37:46.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.513+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a4009b30 con 0x7f27ac0b3090 2023-10-19T10:37:46.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.513+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c00cbd0 con 0x7f27ac0b3090 2023-10-19T10:37:46.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.513+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00eee0 con 0x7f27ac0b3090 2023-10-19T10:37:46.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.513+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac0cbcd0 con 0x7f27ac0b3090 2023-10-19T10:37:46.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.514+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac04e420 con 0x7f27ac0b3090 2023-10-19T10:37:46.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.514+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f279c005990 con 0x7f27ac0b3090 2023-10-19T10:37:46.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.515+0000 7f27a2ffd640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f278c06a080 0x7f278c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.515+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f279c088ed0 con 0x7f27ac0b3090 2023-10-19T10:37:46.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.515+0000 7f27a3fff640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f278c06a080 0x7f278c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.516+0000 7f27a3fff640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f278c06a080 0x7f278c06c540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2794007fd0 tx=0x7f2794009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.519+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f279c054170 con 0x7f27ac0b3090 2023-10-19T10:37:46.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.693+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27ac0b86b0 con 0x7f27ac0b3090 2023-10-19T10:37:46.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.693+0000 7f27a2ffd640 1 -- 172.21.15.50:0/3533315542 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f279c057790 con 0x7f27ac0b3090 2023-10-19T10:37:46.696 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:46.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f278c06a080 msgr2=0x7f278c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f278c06a080 0x7f278c06c540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2794007fd0 tx=0x7f2794009040 comp rx=0 tx=0).stop 2023-10-19T10:37:46.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 msgr2=0x7f27ac156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 0x7f27ac156af0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f279c00ebe0 tx=0x7f279c00c520 comp rx=0 tx=0).stop 2023-10-19T10:37:46.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 shutdown_connections 2023-10-19T10:37:46.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f278c06a080 0x7f278c06c540 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27ac161210 0x7f27ac0b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27ac0b38f0 0x7f27ac0b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.695+0000 7f27b0c46640 1 --2- 172.21.15.50:0/3533315542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27ac0b3090 0x7f27ac156af0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.696+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 >> 172.21.15.50:0/3533315542 conn(0x7f27ac09b960 msgr2=0x7f27ac15d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:46.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.696+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 shutdown_connections 2023-10-19T10:37:46.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.696+0000 7f27b0c46640 1 -- 172.21.15.50:0/3533315542 wait complete. 2023-10-19T10:37:46.709 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:46.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.816+0000 7f360d1c7640 1 Processor -- start 2023-10-19T10:37:46.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.816+0000 7f360d1c7640 1 -- start start 2023-10-19T10:37:46.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.816+0000 7f360d1c7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3608160d70 0x7f360815d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.816+0000 7f360d1c7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360815d8e0 con 0x7f3608145640 2023-10-19T10:37:46.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.816+0000 7f360d1c7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3608160290 con 0x7f36080b2420 2023-10-19T10:37:46.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.816+0000 7f360d1c7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3608160570 con 0x7f3608160d70 2023-10-19T10:37:46.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f360759e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3608160d70 0x7f360815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f3606d9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3608145640 0x7f36080b1d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44672/0 (socket says 172.21.15.50:44672) 2023-10-19T10:37:46.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f3606d9d640 1 -- 172.21.15.50:0/2254354562 learned_addr learned my addr 172.21.15.50:0/2254354562 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:46.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f360759e640 1 -- 172.21.15.50:0/2254354562 >> v1:172.21.15.183:6789/0 conn(0x7f36080b2420 legacy=0x7f360815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:46.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f360759e640 1 -- 172.21.15.50:0/2254354562 >> v1:172.21.15.50:6789/0 conn(0x7f3608145640 legacy=0x7f36080b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f360759e640 1 -- 172.21.15.50:0/2254354562 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608160850 con 0x7f3608160d70 2023-10-19T10:37:46.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.817+0000 7f360659c640 1 -- 172.21.15.50:0/2254354562 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207919830 0 0) 0x7f360815d8e0 con 0x7f3608145640 2023-10-19T10:37:46.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360759e640 1 --2- 172.21.15.50:0/2254354562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360815d250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3600009d90 tx=0x7f360002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c441b018585471 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360659c640 1 -- 172.21.15.50:0/2254354562 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360002cc60 con 0x7f3608160d70 2023-10-19T10:37:46.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360659c640 1 -- 172.21.15.50:0/2254354562 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360002cdc0 con 0x7f3608160d70 2023-10-19T10:37:46.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360659c640 1 -- 172.21.15.50:0/2254354562 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600034660 con 0x7f3608160d70 2023-10-19T10:37:46.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 -- 172.21.15.50:0/2254354562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 msgr2=0x7f360815d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 --2- 172.21.15.50:0/2254354562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360815d250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3600009d90 tx=0x7f360002beb0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 -- 172.21.15.50:0/2254354562 shutdown_connections 2023-10-19T10:37:46.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 --2- 172.21.15.50:0/2254354562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360815d250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 -- 172.21.15.50:0/2254354562 >> 172.21.15.50:0/2254354562 conn(0x7f36080ccd10 msgr2=0x7f36080cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:46.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 -- 172.21.15.50:0/2254354562 shutdown_connections 2023-10-19T10:37:46.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.818+0000 7f360d1c7640 1 -- 172.21.15.50:0/2254354562 wait complete. 2023-10-19T10:37:46.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.820+0000 7f360d1c7640 1 Processor -- start 2023-10-19T10:37:46.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.820+0000 7f360d1c7640 1 -- start start 2023-10-19T10:37:46.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.820+0000 7f360d1c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36080b2420 0x7f36080c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360d1c7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3608145640 0x7f36080c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360d1c7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360806be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360d1c7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3608159b50 con 0x7f3608160d70 2023-10-19T10:37:46.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360d1c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36080c2a60 con 0x7f36080b2420 2023-10-19T10:37:46.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360d1c7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36080c41b0 con 0x7f3608145640 2023-10-19T10:37:46.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360806be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360806be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32894/0 (socket says 172.21.15.50:32894) 2023-10-19T10:37:46.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 -- 172.21.15.50:0/4021411712 learned_addr learned my addr 172.21.15.50:0/4021411712 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:46.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f3606d9d640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36080b2420 0x7f36080c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f35fffff640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3608145640 0x7f36080c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 -- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3608145640 msgr2=0x7f36080c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3608145640 0x7f36080c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 -- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36080b2420 msgr2=0x7f36080c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36080b2420 0x7f36080c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 -- 172.21.15.50:0/4021411712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600009a70 con 0x7f3608160d70 2023-10-19T10:37:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f35fffff640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3608145640 0x7f36080c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360759e640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360806be20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3600009eb0 tx=0x7f3600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360003f420 con 0x7f3608160d70 2023-10-19T10:37:46.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360003f580 con 0x7f3608160d70 2023-10-19T10:37:46.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36080c9d20 con 0x7f3608160d70 2023-10-19T10:37:46.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.821+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360003f910 con 0x7f3608160d70 2023-10-19T10:37:46.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.822+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36080ca2e0 con 0x7f3608160d70 2023-10-19T10:37:46.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.823+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360804e420 con 0x7f3608160d70 2023-10-19T10:37:46.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.823+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f360003fa70 con 0x7f3608160d70 2023-10-19T10:37:46.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.823+0000 7f35ff7fe640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f35e806a050 0x7f35e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:46.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.823+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f36000ab1e0 con 0x7f3608160d70 2023-10-19T10:37:46.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.824+0000 7f3606d9d640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f35e806a050 0x7f35e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:46.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.824+0000 7f3606d9d640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f35e806a050 0x7f35e806c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35f8000f60 tx=0x7f35f8007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:46.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:46.828+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3600076650 con 0x7f3608160d70 2023-10-19T10:37:47.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.002+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36080c42f0 con 0x7f3608160d70 2023-10-19T10:37:47.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.003+0000 7f35ff7fe640 1 -- 172.21.15.50:0/4021411712 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3600079c70 con 0x7f3608160d70 2023-10-19T10:37:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:47.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f35e806a050 msgr2=0x7f35e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:47.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f35e806a050 0x7f35e806c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35f8000f60 tx=0x7f35f8007d50 comp rx=0 tx=0).stop 2023-10-19T10:37:47.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 msgr2=0x7f360806be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:47.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360806be20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3600009eb0 tx=0x7f3600004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:47.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 shutdown_connections 2023-10-19T10:37:47.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f35e806a050 0x7f35e806c510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3608160d70 0x7f360806be20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3608145640 0x7f36080c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 --2- 172.21.15.50:0/4021411712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36080b2420 0x7f36080c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 >> 172.21.15.50:0/4021411712 conn(0x7f36080ccd10 msgr2=0x7f36080b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:47.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 shutdown_connections 2023-10-19T10:37:47.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.005+0000 7f360d1c7640 1 -- 172.21.15.50:0/4021411712 wait complete. 2023-10-19T10:37:47.026 INFO:tasks.cephfs_test_runner:Starting test: test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) Simulate software freeze or temporary network failure. 2023-10-19T10:37:47.027 INFO:tasks.cephfs_test_runner:test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) 2023-10-19T10:37:47.028 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death' 2023-10-19T10:37:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.132+0000 7f1fdc31f640 1 Processor -- start 2023-10-19T10:37:47.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.132+0000 7f1fdc31f640 1 -- start start 2023-10-19T10:37:47.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fdc31f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fd40cd300 0x7f1fd415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fdc31f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1fd40cd6e0 0x7f1fd415c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fdc31f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40b7fc0 0x7f1fd40d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fdc31f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40b7960 con 0x7f1fd40cd300 2023-10-19T10:37:47.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fdc31f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40b6de0 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fdc31f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40b70c0 con 0x7f1fd40b7fc0 2023-10-19T10:37:47.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fda0bb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fd40cd300 0x7f1fd415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fda0bb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fd40cd300 0x7f1fd415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32902/0 (socket says 172.21.15.50:32902) 2023-10-19T10:37:47.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fda0bb640 1 -- 172.21.15.50:0/2061676807 learned_addr learned my addr 172.21.15.50:0/2061676807 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fd98ba640 1 --2- 172.21.15.50:0/2061676807 >> v2:172.21.15.183:3300/0 conn(0x7f1fd40cd6e0 0x7f1fd415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fda8bc640 1 --2- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40b7fc0 0x7f1fd40d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fd98ba640 1 -- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40b7fc0 msgr2=0x7f1fd40d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fd98ba640 1 --2- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40b7fc0 0x7f1fd40d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fd98ba640 1 -- 172.21.15.50:0/2061676807 >> v2:172.21.15.50:3300/0 conn(0x7f1fd40cd300 msgr2=0x7f1fd415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fd98ba640 1 --2- 172.21.15.50:0/2061676807 >> v2:172.21.15.50:3300/0 conn(0x7f1fd40cd300 0x7f1fd415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.133+0000 7f1fd98ba640 1 -- 172.21.15.50:0/2061676807 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40b73a0 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.134+0000 7f1fd98ba640 1 --2- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd415c530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40d3490 tx=0x7f1fc402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afc73e71c4a6bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:47.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.134+0000 7f1fd90b9640 1 -- 172.21.15.50:0/2061676807 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4031830 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.134+0000 7f1fd90b9640 1 -- 172.21.15.50:0/2061676807 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4031990 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.134+0000 7f1fd90b9640 1 -- 172.21.15.50:0/2061676807 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4031d20 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 -- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40cd6e0 msgr2=0x7f1fd415c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd415c530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40d3490 tx=0x7f1fc402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 -- 172.21.15.50:0/2061676807 shutdown_connections 2023-10-19T10:37:47.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40b7fc0 0x7f1fd40d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/2061676807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd415c530 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/2061676807 >> v2:172.21.15.50:3300/0 conn(0x7f1fd40cd300 0x7f1fd415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 -- 172.21.15.50:0/2061676807 >> 172.21.15.50:0/2061676807 conn(0x7f1fd4071320 msgr2=0x7f1fd4071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:47.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 -- 172.21.15.50:0/2061676807 shutdown_connections 2023-10-19T10:37:47.144 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.135+0000 7f1fdc31f640 1 -- 172.21.15.50:0/2061676807 wait complete. 2023-10-19T10:37:47.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.136+0000 7f1fdc31f640 1 Processor -- start 2023-10-19T10:37:47.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.136+0000 7f1fdc31f640 1 -- start start 2023-10-19T10:37:47.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fdc31f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40b7fc0 0x7f1fd40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fdc31f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40cd300 0x7f1fd40d2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fdc31f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd40cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fdc31f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4149c10 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fdc31f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4149600 con 0x7f1fd40b7fc0 2023-10-19T10:37:47.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fdc31f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd415ea80 con 0x7f1fd40cd300 2023-10-19T10:37:47.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd40cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd40cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32908/0 (socket says 172.21.15.50:32908) 2023-10-19T10:37:47.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 -- 172.21.15.50:0/3085478368 learned_addr learned my addr 172.21.15.50:0/3085478368 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:47.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda0bb640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40b7fc0 0x7f1fd40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fd98ba640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40cd300 0x7f1fd40d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 -- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40cd300 msgr2=0x7f1fd40d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:47.148 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40cd300 0x7f1fd40d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 -- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40b7fc0 msgr2=0x7f1fd40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:47.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40b7fc0 0x7f1fd40d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:47.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fda8bc640 1 -- 172.21.15.50:0/3085478368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4009400 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.137+0000 7f1fd98ba640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40cd300 0x7f1fd40d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:47.149 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.138+0000 7f1fda8bc640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd40cba10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00efc0 tx=0x7f1fcc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:47.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.138+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc014480 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.138+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc00f040 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.138+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd41498a0 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.138+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc014820 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.138+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd40cbf50 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.139+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd4073990 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.139+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1fcc014980 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.139+0000 7f1fcaffd640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fb806a050 0x7f1fb806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:47.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.140+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1fcc089410 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.140+0000 7f1fda0bb640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fb806a050 0x7f1fb806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:47.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.140+0000 7f1fda0bb640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fb806a050 0x7f1fb806c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009b50 tx=0x7f1fd0009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:47.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.144+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1fcc054750 con 0x7f1fd40cd6e0 2023-10-19T10:37:47.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:47.392+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]} v 0) v1 -- 0x7f1fd40d00f0 con 0x7f1fd40cd6e0 2023-10-19T10:37:48.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.247+0000 7f1fcaffd640 1 -- 172.21.15.50:0/3085478368 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]}]=0 v72) v1 ==== 160+0+0 (secure 0 0 0) 0x7f1fcc057d70 con 0x7f1fd40cd6e0 2023-10-19T10:37:48.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.249+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fb806a050 msgr2=0x7f1fb806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.249+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fb806a050 0x7f1fb806c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009b50 tx=0x7f1fd0009240 comp rx=0 tx=0).stop 2023-10-19T10:37:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.249+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 msgr2=0x7f1fd40cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.249+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd40cba10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00efc0 tx=0x7f1fcc00c490 comp rx=0 tx=0).stop 2023-10-19T10:37:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 shutdown_connections 2023-10-19T10:37:48.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fb806a050 0x7f1fb806c510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fd40cd6e0 0x7f1fd40cba10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fd40cd300 0x7f1fd40d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 --2- 172.21.15.50:0/3085478368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd40b7fc0 0x7f1fd40d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 >> 172.21.15.50:0/3085478368 conn(0x7f1fd4071320 msgr2=0x7f1fd40d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:48.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 shutdown_connections 2023-10-19T10:37:48.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.250+0000 7f1fdc31f640 1 -- 172.21.15.50:0/3085478368 wait complete. 2023-10-19T10:37:48.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:48.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 Processor -- start 2023-10-19T10:37:48.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 -- start start 2023-10-19T10:37:48.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d40d0e30 0x7f52d415c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52d40cd920 0x7f52d40d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d406aa30 con 0x7f52d415c570 2023-10-19T10:37:48.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d415ce90 con 0x7f52d40cd920 2023-10-19T10:37:48.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.376+0000 7f52d8b43640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d406b2d0 con 0x7f52d40d0e30 2023-10-19T10:37:48.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d2d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52d40cd920 0x7f52d40d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d259c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f52d415c570 0x7f52d406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44686/0 (socket says 172.21.15.50:44686) 2023-10-19T10:37:48.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d259c640 1 -- 172.21.15.50:0/3680529803 learned_addr learned my addr 172.21.15.50:0/3680529803 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:48.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52c9d9b640 1 --2- 172.21.15.50:0/3680529803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d40d0e30 0x7f52d415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d2d9d640 1 -- 172.21.15.50:0/3680529803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d40d0e30 msgr2=0x7f52d415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d2d9d640 1 --2- 172.21.15.50:0/3680529803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d40d0e30 0x7f52d415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d2d9d640 1 -- 172.21.15.50:0/3680529803 >> v1:172.21.15.50:6789/0 conn(0x7f52d415c570 legacy=0x7f52d406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d1d9b640 1 -- 172.21.15.50:0/3680529803 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302521478 0 0) 0x7f52d406aa30 con 0x7f52d415c570 2023-10-19T10:37:48.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.377+0000 7f52d2d9d640 1 -- 172.21.15.50:0/3680529803 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40d1270 con 0x7f52d40cd920 2023-10-19T10:37:48.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d2d9d640 1 --2- 172.21.15.50:0/3680529803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d40d26a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009d90 tx=0x7f52cc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2640f5b64c84568 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:48.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d1d9b640 1 -- 172.21.15.50:0/3680529803 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc02cbc0 con 0x7f52d40cd920 2023-10-19T10:37:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d1d9b640 1 -- 172.21.15.50:0/3680529803 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc02cd20 con 0x7f52d40cd920 2023-10-19T10:37:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d1d9b640 1 -- 172.21.15.50:0/3680529803 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc034660 con 0x7f52d40cd920 2023-10-19T10:37:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 -- 172.21.15.50:0/3680529803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 msgr2=0x7f52d40d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 --2- 172.21.15.50:0/3680529803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d40d26a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009d90 tx=0x7f52cc02be10 comp rx=0 tx=0).stop 2023-10-19T10:37:48.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 -- 172.21.15.50:0/3680529803 shutdown_connections 2023-10-19T10:37:48.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 --2- 172.21.15.50:0/3680529803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d40d26a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 --2- 172.21.15.50:0/3680529803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d40d0e30 0x7f52d415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 -- 172.21.15.50:0/3680529803 >> 172.21.15.50:0/3680529803 conn(0x7f52d4089ff0 msgr2=0x7f52d409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:48.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 -- 172.21.15.50:0/3680529803 shutdown_connections 2023-10-19T10:37:48.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.378+0000 7f52d8b43640 1 -- 172.21.15.50:0/3680529803 wait complete. 2023-10-19T10:37:48.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.380+0000 7f52d8b43640 1 Processor -- start 2023-10-19T10:37:48.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.380+0000 7f52d8b43640 1 -- start start 2023-10-19T10:37:48.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d41f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52d40d0e30 0x7f52d40b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d415c570 0x7f52d40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d415cfd0 con 0x7f52d40cd920 2023-10-19T10:37:48.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4150870 con 0x7f52d40d0e30 2023-10-19T10:37:48.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41e6970 con 0x7f52d415c570 2023-10-19T10:37:48.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d41f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d41f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32938/0 (socket says 172.21.15.50:32938) 2023-10-19T10:37:48.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 -- 172.21.15.50:0/2813541462 learned_addr learned my addr 172.21.15.50:0/2813541462 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52c9d9b640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52d40d0e30 0x7f52d40b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d2d9d640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d415c570 0x7f52d40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 -- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d415c570 msgr2=0x7f52d40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d415c570 0x7f52d40b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 -- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52d40d0e30 msgr2=0x7f52d40b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52d40d0e30 0x7f52d40b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 -- 172.21.15.50:0/2813541462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc009a70 con 0x7f52d40cd920 2023-10-19T10:37:48.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d2d9d640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d415c570 0x7f52d40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:48.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52c9d9b640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52d40d0e30 0x7f52d40b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:48.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d259c640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d41f55f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c500 tx=0x7f52c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:48.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4007480 con 0x7f52d40cd920 2023-10-19T10:37:48.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40075e0 con 0x7f52d40cd920 2023-10-19T10:37:48.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.381+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41e5a80 con 0x7f52d40cd920 2023-10-19T10:37:48.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.382+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40059a0 con 0x7f52d40cd920 2023-10-19T10:37:48.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.382+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41e6040 con 0x7f52d40cd920 2023-10-19T10:37:48.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.383+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d4073990 con 0x7f52d40cd920 2023-10-19T10:37:48.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.383+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52c4005b00 con 0x7f52d40cd920 2023-10-19T10:37:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.383+0000 7f52cb7fe640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f52b406a020 0x7f52b406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.383+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f52c40884c0 con 0x7f52d40cd920 2023-10-19T10:37:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.384+0000 7f52c9d9b640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f52b406a020 0x7f52b406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.384+0000 7f52c9d9b640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f52b406a020 0x7f52b406c4e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52bc006fd0 tx=0x7f52bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:48.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.388+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f52c4053880 con 0x7f52d40cd920 2023-10-19T10:37:48.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.564+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d40d3450 con 0x7f52d40cd920 2023-10-19T10:37:48.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.565+0000 7f52cb7fe640 1 -- 172.21.15.50:0/2813541462 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f52c4005db0 con 0x7f52d40cd920 2023-10-19T10:37:48.568 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:48.568 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:48.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f52b406a020 msgr2=0x7f52b406c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f52b406a020 0x7f52b406c4e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52bc006fd0 tx=0x7f52bc008040 comp rx=0 tx=0).stop 2023-10-19T10:37:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 msgr2=0x7f52d41f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d41f55f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c500 tx=0x7f52c400c9c0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 shutdown_connections 2023-10-19T10:37:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f52b406a020 0x7f52b406c4e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52d415c570 0x7f52d40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52d40d0e30 0x7f52d40b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 --2- 172.21.15.50:0/2813541462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d40cd920 0x7f52d41f55f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 >> 172.21.15.50:0/2813541462 conn(0x7f52d4089ff0 msgr2=0x7f52d415bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:48.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 shutdown_connections 2023-10-19T10:37:48.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.567+0000 7f52d8b43640 1 -- 172.21.15.50:0/2813541462 wait complete. 2023-10-19T10:37:48.580 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:48.581 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:48.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.688+0000 7f1f58677640 1 Processor -- start 2023-10-19T10:37:48.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.688+0000 7f1f58677640 1 -- start start 2023-10-19T10:37:48.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f58677640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1f500b38f0 0x7f1f500d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f58677640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1f500b3090 0x7f1f50160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f58677640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f500af250 con 0x7f1f500b3cd0 2023-10-19T10:37:48.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f58677640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f500b0190 con 0x7f1f500b38f0 2023-10-19T10:37:48.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f58677640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f500af510 con 0x7f1f500b3090 2023-10-19T10:37:48.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56413640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1f500b38f0 0x7f1f500d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56c14640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1f500b3cd0 0x7f1f500b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44694/0 (socket says 172.21.15.50:44694) 2023-10-19T10:37:48.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56413640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1f500b38f0 0x7f1f500d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32942/0 (socket says 172.21.15.50:32942) 2023-10-19T10:37:48.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56c14640 1 -- 172.21.15.50:0/3782450720 learned_addr learned my addr 172.21.15.50:0/3782450720 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f55c12640 1 --2- 172.21.15.50:0/3782450720 >> v2:172.21.15.183:3300/0 conn(0x7f1f500b3090 0x7f1f50160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56413640 1 -- 172.21.15.50:0/3782450720 >> v2:172.21.15.183:3300/0 conn(0x7f1f500b3090 msgr2=0x7f1f50160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56413640 1 --2- 172.21.15.50:0/3782450720 >> v2:172.21.15.183:3300/0 conn(0x7f1f500b3090 0x7f1f50160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56413640 1 -- 172.21.15.50:0/3782450720 >> v1:172.21.15.50:6789/0 conn(0x7f1f500b3cd0 legacy=0x7f1f500b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.689+0000 7f1f56413640 1 -- 172.21.15.50:0/3782450720 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500af7f0 con 0x7f1f500b38f0 2023-10-19T10:37:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.690+0000 7f1f56413640 1 --2- 172.21.15.50:0/3782450720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b38f0 0x7f1f500d30e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f500c6230 tx=0x7f1f4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62113f4c2c61dcce server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.690+0000 7f1f55411640 1 -- 172.21.15.50:0/3782450720 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c031830 con 0x7f1f500b38f0 2023-10-19T10:37:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.690+0000 7f1f55411640 1 -- 172.21.15.50:0/3782450720 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c031990 con 0x7f1f500b38f0 2023-10-19T10:37:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.690+0000 7f1f55411640 1 -- 172.21.15.50:0/3782450720 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c031d20 con 0x7f1f500b38f0 2023-10-19T10:37:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 -- 172.21.15.50:0/3782450720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b38f0 msgr2=0x7f1f500d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 --2- 172.21.15.50:0/3782450720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b38f0 0x7f1f500d30e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f500c6230 tx=0x7f1f4c02b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 -- 172.21.15.50:0/3782450720 shutdown_connections 2023-10-19T10:37:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 --2- 172.21.15.50:0/3782450720 >> v2:172.21.15.183:3300/0 conn(0x7f1f500b3090 0x7f1f50160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 --2- 172.21.15.50:0/3782450720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b38f0 0x7f1f500d30e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 -- 172.21.15.50:0/3782450720 >> 172.21.15.50:0/3782450720 conn(0x7f1f5009b960 msgr2=0x7f1f500d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 -- 172.21.15.50:0/3782450720 shutdown_connections 2023-10-19T10:37:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.691+0000 7f1f58677640 1 -- 172.21.15.50:0/3782450720 wait complete. 2023-10-19T10:37:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.692+0000 7f1f58677640 1 Processor -- start 2023-10-19T10:37:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.692+0000 7f1f58677640 1 -- start start 2023-10-19T10:37:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 0x7f1f50154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f500b38f0 0x7f1f500ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f500b3cd0 0x7f1f5015f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500ce280 con 0x7f1f500b3090 2023-10-19T10:37:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f50155440 con 0x7f1f500b38f0 2023-10-19T10:37:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f50159f50 con 0x7f1f500b3cd0 2023-10-19T10:37:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 0x7f1f50154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 0x7f1f50154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32944/0 (socket says 172.21.15.50:32944) 2023-10-19T10:37:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 -- 172.21.15.50:0/1467156876 learned_addr learned my addr 172.21.15.50:0/1467156876 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f55c12640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f500b38f0 0x7f1f500ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56c14640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f500b3cd0 0x7f1f5015f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 -- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f500b3cd0 msgr2=0x7f1f5015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f500b3cd0 0x7f1f5015f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 -- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f500b38f0 msgr2=0x7f1f500ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f500b38f0 0x7f1f500ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 -- 172.21.15.50:0/1467156876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0093a0 con 0x7f1f500b3090 2023-10-19T10:37:48.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f55c12640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f500b38f0 0x7f1f500ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:48.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f56413640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 0x7f1f50154f00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009880 tx=0x7f1f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c03ba70 con 0x7f1f500b3090 2023-10-19T10:37:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c03bbd0 con 0x7f1f500b3090 2023-10-19T10:37:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f50159c20 con 0x7f1f500b3090 2023-10-19T10:37:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.693+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c005990 con 0x7f1f500b3090 2023-10-19T10:37:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.694+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f500cdcc0 con 0x7f1f500b3090 2023-10-19T10:37:48.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.695+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5004e420 con 0x7f1f500b3090 2023-10-19T10:37:48.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.695+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1f4c04e440 con 0x7f1f500b3090 2023-10-19T10:37:48.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.695+0000 7f1f46ffd640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f3406a050 0x7f1f3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:48.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.695+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1f4c0aab60 con 0x7f1f500b3090 2023-10-19T10:37:48.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.695+0000 7f1f55c12640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f3406a050 0x7f1f3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:48.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.696+0000 7f1f55c12640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f3406a050 0x7f1f3406c510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ce600 tx=0x7f1f40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:48.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.699+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1f4c076040 con 0x7f1f500b3090 2023-10-19T10:37:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.873+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f500b86b0 con 0x7f1f500b3090 2023-10-19T10:37:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.873+0000 7f1f46ffd640 1 -- 172.21.15.50:0/1467156876 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f4c0350a0 con 0x7f1f500b3090 2023-10-19T10:37:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f3406a050 msgr2=0x7f1f3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f3406a050 0x7f1f3406c510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ce600 tx=0x7f1f40008040 comp rx=0 tx=0).stop 2023-10-19T10:37:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 msgr2=0x7f1f50154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 0x7f1f50154f00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009880 tx=0x7f1f4c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 shutdown_connections 2023-10-19T10:37:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f3406a050 0x7f1f3406c510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f500b3cd0 0x7f1f5015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f500b38f0 0x7f1f500ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 --2- 172.21.15.50:0/1467156876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f500b3090 0x7f1f50154f00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:48.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 >> 172.21.15.50:0/1467156876 conn(0x7f1f5009b960 msgr2=0x7f1f50155640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:48.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 shutdown_connections 2023-10-19T10:37:48.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.875+0000 7f1f58677640 1 -- 172.21.15.50:0/1467156876 wait complete. 2023-10-19T10:37:48.889 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:48.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.995+0000 7f662c7a0640 1 Processor -- start 2023-10-19T10:37:48.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.995+0000 7f662c7a0640 1 -- start start 2023-10-19T10:37:48.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f662c7a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66240ce930 0x7f66240cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f662c7a0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66240d0fa0 0x7f66240cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f662c7a0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66240ced10 con 0x7f662415cf10 2023-10-19T10:37:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f662c7a0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662415dc40 con 0x7f66240d0fa0 2023-10-19T10:37:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f662c7a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66240d04c0 con 0x7f66240ce930 2023-10-19T10:37:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66240d0fa0 0x7f66240cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66240d0fa0 0x7f66240cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32956/0 (socket says 172.21.15.50:32956) 2023-10-19T10:37:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 -- 172.21.15.50:0/1586536012 learned_addr learned my addr 172.21.15.50:0/1586536012 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f662a53c640 1 --2- 172.21.15.50:0/1586536012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66240ce930 0x7f66240cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 -- 172.21.15.50:0/1586536012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66240ce930 msgr2=0x7f66240cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 --2- 172.21.15.50:0/1586536012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66240ce930 0x7f66240cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 -- 172.21.15.50:0/1586536012 >> v1:172.21.15.183:6789/0 conn(0x7f662415cf10 legacy=0x7f662415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.996+0000 7f6629d3b640 1 -- 172.21.15.50:0/1586536012 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240d07a0 con 0x7f66240d0fa0 2023-10-19T10:37:49.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.997+0000 7f6629d3b640 1 --2- 172.21.15.50:0/1586536012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f66240cd3d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6614009870 tx=0x7f661402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289ed0a582adad4e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.997+0000 7f662953a640 1 -- 172.21.15.50:0/1586536012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614031640 con 0x7f66240d0fa0 2023-10-19T10:37:49.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.997+0000 7f662953a640 1 -- 172.21.15.50:0/1586536012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66140317a0 con 0x7f66240d0fa0 2023-10-19T10:37:49.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.997+0000 7f662953a640 1 -- 172.21.15.50:0/1586536012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614031b30 con 0x7f66240d0fa0 2023-10-19T10:37:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.997+0000 7f662c7a0640 1 -- 172.21.15.50:0/1586536012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 msgr2=0x7f66240cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.997+0000 7f662c7a0640 1 --2- 172.21.15.50:0/1586536012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f66240cd3d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6614009870 tx=0x7f661402b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.998+0000 7f662c7a0640 1 -- 172.21.15.50:0/1586536012 shutdown_connections 2023-10-19T10:37:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.998+0000 7f662c7a0640 1 --2- 172.21.15.50:0/1586536012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f66240cd3d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.998+0000 7f662c7a0640 1 --2- 172.21.15.50:0/1586536012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66240ce930 0x7f66240cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.998+0000 7f662c7a0640 1 -- 172.21.15.50:0/1586536012 >> 172.21.15.50:0/1586536012 conn(0x7f6624069800 msgr2=0x7f6624069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.998+0000 7f662c7a0640 1 -- 172.21.15.50:0/1586536012 shutdown_connections 2023-10-19T10:37:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.998+0000 7f662c7a0640 1 -- 172.21.15.50:0/1586536012 wait complete. 2023-10-19T10:37:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.999+0000 7f662c7a0640 1 Processor -- start 2023-10-19T10:37:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.999+0000 7f662c7a0640 1 -- start start 2023-10-19T10:37:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.999+0000 7f662c7a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66240ce930 0x7f66241f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.999+0000 7f662c7a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f662415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.999+0000 7f662c7a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f662415cf10 0x7f66241409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:48.999+0000 7f662c7a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662413f950 con 0x7f66240d0fa0 2023-10-19T10:37:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f662c7a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662413fc30 con 0x7f66240ce930 2023-10-19T10:37:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f662c7a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662413ff10 con 0x7f662415cf10 2023-10-19T10:37:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f662415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f662415df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32970/0 (socket says 172.21.15.50:32970) 2023-10-19T10:37:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 -- 172.21.15.50:0/4268448385 learned_addr learned my addr 172.21.15.50:0/4268448385 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f662a53c640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66240ce930 0x7f66241f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f662ad3d640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f662415cf10 0x7f66241409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 -- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f662415cf10 msgr2=0x7f66241409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f662415cf10 0x7f66241409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 -- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66240ce930 msgr2=0x7f66241f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66240ce930 0x7f66241f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 -- 172.21.15.50:0/4268448385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140094d0 con 0x7f66240d0fa0 2023-10-19T10:37:49.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f6629d3b640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f662415df60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6614000c00 tx=0x7f66140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661403ac60 con 0x7f66240d0fa0 2023-10-19T10:37:49.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661403adc0 con 0x7f66240d0fa0 2023-10-19T10:37:49.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66241401f0 con 0x7f66240d0fa0 2023-10-19T10:37:49.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614044630 con 0x7f66240d0fa0 2023-10-19T10:37:49.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.000+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6624152cb0 con 0x7f66240d0fa0 2023-10-19T10:37:49.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.001+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662404e420 con 0x7f66240d0fa0 2023-10-19T10:37:49.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.002+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f661402f050 con 0x7f66240d0fa0 2023-10-19T10:37:49.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.002+0000 7f661affd640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f660806a050 0x7f660806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.002+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f66140aa7a0 con 0x7f66240d0fa0 2023-10-19T10:37:49.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.002+0000 7f662a53c640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f660806a050 0x7f660806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.003+0000 7f662a53c640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f660806a050 0x7f660806c510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6620006fd0 tx=0x7f6620008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.007+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6614075ac0 con 0x7f66240d0fa0 2023-10-19T10:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.182+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f66240b60e0 con 0x7f66240d0fa0 2023-10-19T10:37:49.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.182+0000 7f661affd640 1 -- 172.21.15.50:0/4268448385 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6614031e60 con 0x7f66240d0fa0 2023-10-19T10:37:49.185 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:49.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f660806a050 msgr2=0x7f660806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f660806a050 0x7f660806c510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6620006fd0 tx=0x7f6620008040 comp rx=0 tx=0).stop 2023-10-19T10:37:49.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 msgr2=0x7f662415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f662415df60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6614000c00 tx=0x7f66140046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 shutdown_connections 2023-10-19T10:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f660806a050 0x7f660806c510 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f662415cf10 0x7f66241409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66240d0fa0 0x7f662415df60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 --2- 172.21.15.50:0/4268448385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66240ce930 0x7f66241f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 >> 172.21.15.50:0/4268448385 conn(0x7f6624069800 msgr2=0x7f662415b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 shutdown_connections 2023-10-19T10:37:49.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.184+0000 7f662c7a0640 1 -- 172.21.15.50:0/4268448385 wait complete. 2023-10-19T10:37:49.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:49.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.304+0000 7fefab08b640 1 Processor -- start 2023-10-19T10:37:49.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.304+0000 7fefab08b640 1 -- start start 2023-10-19T10:37:49.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefab08b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 0x7fefa40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefab08b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefab08b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa40b4990 con 0x7fefa415cdf0 2023-10-19T10:37:49.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefab08b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa415d280 con 0x7fefa40b4070 2023-10-19T10:37:49.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefab08b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa415d560 con 0x7fefa415c960 2023-10-19T10:37:49.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefa8e27640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 0x7fefa40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefa3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefa3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:60620/0 (socket says 172.21.15.50:60620) 2023-10-19T10:37:49.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefa8e27640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 0x7fefa40b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:44360/0 (socket says 172.21.15.50:44360) 2023-10-19T10:37:49.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefa3fff640 1 -- 172.21.15.50:0/3734806466 learned_addr learned my addr 172.21.15.50:0/3734806466 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.305+0000 7fefa9628640 1 --1- 172.21.15.50:0/3734806466 >> v1:172.21.15.183:6789/0 conn(0x7fefa415cdf0 0x7fefa4160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:59926/0 (socket says 172.21.15.50:59926) 2023-10-19T10:37:49.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa37fe640 1 -- 172.21.15.50:0/3734806466 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395651038 0 0) 0x7fefa40b4990 con 0x7fefa415cdf0 2023-10-19T10:37:49.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa37fe640 1 -- 172.21.15.50:0/3734806466 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef8c003520 con 0x7fefa415cdf0 2023-10-19T10:37:49.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa3fff640 1 -- 172.21.15.50:0/3734806466 >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 msgr2=0x7fefa40b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:49.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa3fff640 1 --2- 172.21.15.50:0/3734806466 >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 0x7fefa40b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa3fff640 1 -- 172.21.15.50:0/3734806466 >> v1:172.21.15.183:6789/0 conn(0x7fefa415cdf0 legacy=0x7fefa4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa3fff640 1 -- 172.21.15.50:0/3734806466 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa415d840 con 0x7fefa415c960 2023-10-19T10:37:49.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.306+0000 7fefa8e27640 1 --2- 172.21.15.50:0/3734806466 >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 0x7fefa40b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:49.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefa3fff640 1 --2- 172.21.15.50:0/3734806466 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40d2d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef98009870 tx=0x7fef9802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b61a96244478e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefa37fe640 1 -- 172.21.15.50:0/3734806466 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98030840 con 0x7fefa415c960 2023-10-19T10:37:49.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefa37fe640 1 -- 172.21.15.50:0/3734806466 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef980309a0 con 0x7fefa415c960 2023-10-19T10:37:49.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefa37fe640 1 -- 172.21.15.50:0/3734806466 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98030d10 con 0x7fefa415c960 2023-10-19T10:37:49.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 -- 172.21.15.50:0/3734806466 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 msgr2=0x7fefa40d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 --2- 172.21.15.50:0/3734806466 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40d2d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef98009870 tx=0x7fef9802b980 comp rx=0 tx=0).stop 2023-10-19T10:37:49.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 -- 172.21.15.50:0/3734806466 shutdown_connections 2023-10-19T10:37:49.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 --2- 172.21.15.50:0/3734806466 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40d2d70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 --2- 172.21.15.50:0/3734806466 >> v2:172.21.15.183:3300/0 conn(0x7fefa40b4070 0x7fefa40b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 -- 172.21.15.50:0/3734806466 >> 172.21.15.50:0/3734806466 conn(0x7fefa409b970 msgr2=0x7fefa40d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 -- 172.21.15.50:0/3734806466 shutdown_connections 2023-10-19T10:37:49.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.307+0000 7fefab08b640 1 -- 172.21.15.50:0/3734806466 wait complete. 2023-10-19T10:37:49.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 Processor -- start 2023-10-19T10:37:49.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 -- start start 2023-10-19T10:37:49.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 0x7fefa40ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415f7c0 0x7fefa41539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40cee20 con 0x7fefa40b4070 2023-10-19T10:37:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40af2c0 con 0x7fefa415f7c0 2023-10-19T10:37:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.309+0000 7fefab08b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40af5a0 con 0x7fefa415c960 2023-10-19T10:37:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 0x7fefa40ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 0x7fefa40ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32986/0 (socket says 172.21.15.50:32986) 2023-10-19T10:37:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 -- 172.21.15.50:0/2122440246 learned_addr learned my addr 172.21.15.50:0/2122440246 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa9628640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415f7c0 0x7fefa41539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa3fff640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 -- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 msgr2=0x7fefa40c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 -- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415f7c0 msgr2=0x7fefa41539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415f7c0 0x7fefa41539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 -- 172.21.15.50:0/2122440246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940093a0 con 0x7fefa40b4070 2023-10-19T10:37:49.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa8e27640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 0x7fefa40ce5d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fef9400ebe0 tx=0x7fef9400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94009750 con 0x7fefa40b4070 2023-10-19T10:37:49.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400cc30 con 0x7fefa40b4070 2023-10-19T10:37:49.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef980094d0 con 0x7fefa40b4070 2023-10-19T10:37:49.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400eee0 con 0x7fefa40b4070 2023-10-19T10:37:49.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.310+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa40afb60 con 0x7fefa40b4070 2023-10-19T10:37:49.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.311+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa404e420 con 0x7fefa40b4070 2023-10-19T10:37:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.312+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fef94005990 con 0x7fefa40b4070 2023-10-19T10:37:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.312+0000 7fefa17fa640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fef9006a050 0x7fef9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.312+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fef94088df0 con 0x7fefa40b4070 2023-10-19T10:37:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.312+0000 7fefa3fff640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fef9006a050 0x7fef9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.313+0000 7fefa3fff640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fef9006a050 0x7fef9006c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef98036000 tx=0x7fef98008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.316+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fef94054260 con 0x7fefa40b4070 2023-10-19T10:37:49.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.495+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefa408c720 con 0x7fefa40b4070 2023-10-19T10:37:49.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.495+0000 7fefa17fa640 1 -- 172.21.15.50:0/2122440246 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef94057880 con 0x7fefa40b4070 2023-10-19T10:37:49.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:49.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fef9006a050 msgr2=0x7fef9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fef9006a050 0x7fef9006c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef98036000 tx=0x7fef98008030 comp rx=0 tx=0).stop 2023-10-19T10:37:49.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 msgr2=0x7fefa40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 0x7fefa40ce5d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fef9400ebe0 tx=0x7fef9400c520 comp rx=0 tx=0).stop 2023-10-19T10:37:49.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 shutdown_connections 2023-10-19T10:37:49.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fef9006a050 0x7fef9006c510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415f7c0 0x7fefa41539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415c960 0x7fefa40c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 --2- 172.21.15.50:0/2122440246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa40b4070 0x7fefa40ce5d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 >> 172.21.15.50:0/2122440246 conn(0x7fefa409b970 msgr2=0x7fefa40cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 shutdown_connections 2023-10-19T10:37:49.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.497+0000 7fefab08b640 1 -- 172.21.15.50:0/2122440246 wait complete. 2023-10-19T10:37:49.510 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.617+0000 7f82d7e9d640 1 Processor -- start 2023-10-19T10:37:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.617+0000 7f82d7e9d640 1 -- start start 2023-10-19T10:37:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d7e9d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f82d015c660 0x7f82d006a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d7e9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f82d00cd460 0x7f82d0070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d7e9d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d0070780 con 0x7f82d0160240 2023-10-19T10:37:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d7e9d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d00cca10 con 0x7f82d00cd460 2023-10-19T10:37:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d7e9d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d006a750 con 0x7f82d015c660 2023-10-19T10:37:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d643a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f82d00cd460 0x7f82d0070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d5438640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f82d0160240 0x7f82d00ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44700/0 (socket says 172.21.15.50:44700) 2023-10-19T10:37:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.618+0000 7f82d5438640 1 -- 172.21.15.50:0/736211385 learned_addr learned my addr 172.21.15.50:0/736211385 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.619+0000 7f82d5c39640 1 --2- 172.21.15.50:0/736211385 >> v2:172.21.15.183:3300/0 conn(0x7f82d015c660 0x7f82d006a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.619+0000 7f82d643a640 1 -- 172.21.15.50:0/736211385 >> v2:172.21.15.183:3300/0 conn(0x7f82d015c660 msgr2=0x7f82d006a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.619+0000 7f82d643a640 1 --2- 172.21.15.50:0/736211385 >> v2:172.21.15.183:3300/0 conn(0x7f82d015c660 0x7f82d006a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.619+0000 7f82d643a640 1 -- 172.21.15.50:0/736211385 >> v1:172.21.15.50:6789/0 conn(0x7f82d0160240 legacy=0x7f82d00ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.619+0000 7f82d643a640 1 -- 172.21.15.50:0/736211385 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0160720 con 0x7f82d00cd460 2023-10-19T10:37:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.619+0000 7f82d4c37640 1 -- 172.21.15.50:0/736211385 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981523388 0 0) 0x7f82d0070780 con 0x7f82d0160240 2023-10-19T10:37:49.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d643a640 1 --2- 172.21.15.50:0/736211385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d0070c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f82d00703f0 tx=0x7f82c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec11dc436119d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d4c37640 1 -- 172.21.15.50:0/736211385 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c002cc80 con 0x7f82d00cd460 2023-10-19T10:37:49.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d4c37640 1 -- 172.21.15.50:0/736211385 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c002cde0 con 0x7f82d00cd460 2023-10-19T10:37:49.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d4c37640 1 -- 172.21.15.50:0/736211385 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0034660 con 0x7f82d00cd460 2023-10-19T10:37:49.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 -- 172.21.15.50:0/736211385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 msgr2=0x7f82d0070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/736211385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d0070c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f82d00703f0 tx=0x7f82c002bed0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 -- 172.21.15.50:0/736211385 shutdown_connections 2023-10-19T10:37:49.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/736211385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d0070c10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/736211385 >> v2:172.21.15.183:3300/0 conn(0x7f82d015c660 0x7f82d006a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 -- 172.21.15.50:0/736211385 >> 172.21.15.50:0/736211385 conn(0x7f82d00cbc00 msgr2=0x7f82d00cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 -- 172.21.15.50:0/736211385 shutdown_connections 2023-10-19T10:37:49.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.620+0000 7f82d7e9d640 1 -- 172.21.15.50:0/736211385 wait complete. 2023-10-19T10:37:49.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.621+0000 7f82d7e9d640 1 Processor -- start 2023-10-19T10:37:49.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 -- start start 2023-10-19T10:37:49.630 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82d015c660 0x7f82d00b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0160240 0x7f82d014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00b4540 con 0x7f82d00cd460 2023-10-19T10:37:49.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00b4820 con 0x7f82d0160240 2023-10-19T10:37:49.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d7e9d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00b4b00 con 0x7f82d015c660 2023-10-19T10:37:49.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d5c39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d5c39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32998/0 (socket says 172.21.15.50:32998) 2023-10-19T10:37:49.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.622+0000 7f82d5c39640 1 -- 172.21.15.50:0/403946275 learned_addr learned my addr 172.21.15.50:0/403946275 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5438640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82d015c660 0x7f82d00b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d643a640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0160240 0x7f82d014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5c39640 1 -- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82d015c660 msgr2=0x7f82d00b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5c39640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82d015c660 0x7f82d00b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5c39640 1 -- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0160240 msgr2=0x7f82d014a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5c39640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0160240 0x7f82d014a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5c39640 1 -- 172.21.15.50:0/403946275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0009a70 con 0x7f82d00cd460 2023-10-19T10:37:49.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d5c39640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d015f2a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f82c800d5f0 tx=0x7f82c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80125f0 con 0x7f82d00cd460 2023-10-19T10:37:49.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c8012750 con 0x7f82d00cd460 2023-10-19T10:37:49.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8012b00 con 0x7f82d00cd460 2023-10-19T10:37:49.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d00b4de0 con 0x7f82d00cd460 2023-10-19T10:37:49.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.623+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d015f7e0 con 0x7f82d00cd460 2023-10-19T10:37:49.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.624+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d00672b0 con 0x7f82d00cd460 2023-10-19T10:37:49.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.625+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f82c8012c60 con 0x7f82d00cd460 2023-10-19T10:37:49.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.625+0000 7f82ce7fc640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82bc06a080 0x7f82bc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.625+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f82c8088cb0 con 0x7f82d00cd460 2023-10-19T10:37:49.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.625+0000 7f82d5438640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82bc06a080 0x7f82bc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.626+0000 7f82d5438640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82bc06a080 0x7f82bc06c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82d006c480 tx=0x7f82c4007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.630+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f82c8054100 con 0x7f82d00cd460 2023-10-19T10:37:49.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.804+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82d004e420 con 0x7f82d00cd460 2023-10-19T10:37:49.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.805+0000 7f82ce7fc640 1 -- 172.21.15.50:0/403946275 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82c8088030 con 0x7f82d00cd460 2023-10-19T10:37:49.808 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:49.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82bc06a080 msgr2=0x7f82bc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82bc06a080 0x7f82bc06c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82d006c480 tx=0x7f82c4007b50 comp rx=0 tx=0).stop 2023-10-19T10:37:49.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 msgr2=0x7f82d015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d015f2a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f82c800d5f0 tx=0x7f82c800dab0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 shutdown_connections 2023-10-19T10:37:49.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f82bc06a080 0x7f82bc06c540 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0160240 0x7f82d014a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82d015c660 0x7f82d00b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 --2- 172.21.15.50:0/403946275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82d00cd460 0x7f82d015f2a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 >> 172.21.15.50:0/403946275 conn(0x7f82d00cbc00 msgr2=0x7f82d0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 shutdown_connections 2023-10-19T10:37:49.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.807+0000 7f82d7e9d640 1 -- 172.21.15.50:0/403946275 wait complete. 2023-10-19T10:37:49.820 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.926+0000 7f053d078640 1 Processor -- start 2023-10-19T10:37:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.926+0000 7f053d078640 1 -- start start 2023-10-19T10:37:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f053d078640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0538069a70 0x7f05380c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f053d078640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05380cb6f0 0x7f053806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f053d078640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380ca120 con 0x7f0538069e50 2023-10-19T10:37:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f053d078640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0538071b90 con 0x7f0538069a70 2023-10-19T10:37:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f053d078640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0538071310 con 0x7f05380cb6f0 2023-10-19T10:37:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f0536d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0538069a70 0x7f05380c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f0536d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0538069a70 0x7f05380c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33006/0 (socket says 172.21.15.50:33006) 2023-10-19T10:37:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f0536d9d640 1 -- 172.21.15.50:0/1146584964 learned_addr learned my addr 172.21.15.50:0/1146584964 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.927+0000 7f053659c640 1 --2- 172.21.15.50:0/1146584964 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05380cb6f0 0x7f053806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.928+0000 7f0536d9d640 1 -- 172.21.15.50:0/1146584964 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05380cb6f0 msgr2=0x7f053806dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:49.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.928+0000 7f0536d9d640 1 --2- 172.21.15.50:0/1146584964 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05380cb6f0 0x7f053806dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.928+0000 7f0536d9d640 1 -- 172.21.15.50:0/1146584964 >> v1:172.21.15.183:6789/0 conn(0x7f0538069e50 legacy=0x7f053806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.928+0000 7f0536d9d640 1 -- 172.21.15.50:0/1146584964 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380715d0 con 0x7f0538069a70 2023-10-19T10:37:49.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.928+0000 7f0536d9d640 1 --2- 172.21.15.50:0/1146584964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069a70 0x7f05380c97d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0530009870 tx=0x7f053002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97b065d71d57128 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f0535d9b640 1 -- 172.21.15.50:0/1146584964 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05300317c0 con 0x7f0538069a70 2023-10-19T10:37:49.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f0535d9b640 1 -- 172.21.15.50:0/1146584964 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0530031920 con 0x7f0538069a70 2023-10-19T10:37:49.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f0535d9b640 1 -- 172.21.15.50:0/1146584964 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530031cb0 con 0x7f0538069a70 2023-10-19T10:37:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 -- 172.21.15.50:0/1146584964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069a70 msgr2=0x7f05380c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 --2- 172.21.15.50:0/1146584964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069a70 0x7f05380c97d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0530009870 tx=0x7f053002b860 comp rx=0 tx=0).stop 2023-10-19T10:37:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 -- 172.21.15.50:0/1146584964 shutdown_connections 2023-10-19T10:37:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 --2- 172.21.15.50:0/1146584964 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05380cb6f0 0x7f053806dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 --2- 172.21.15.50:0/1146584964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069a70 0x7f05380c97d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 -- 172.21.15.50:0/1146584964 >> 172.21.15.50:0/1146584964 conn(0x7f05380cbc00 msgr2=0x7f05380cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 -- 172.21.15.50:0/1146584964 shutdown_connections 2023-10-19T10:37:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.929+0000 7f053d078640 1 -- 172.21.15.50:0/1146584964 wait complete. 2023-10-19T10:37:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 Processor -- start 2023-10-19T10:37:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 -- start start 2023-10-19T10:37:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0538069a70 0x7f0538151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 0x7f05380cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05380cb6f0 0x7f053814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380cdca0 con 0x7f0538069e50 2023-10-19T10:37:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05381f6520 con 0x7f05380cb6f0 2023-10-19T10:37:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053d078640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053814fad0 con 0x7f0538069a70 2023-10-19T10:37:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 0x7f05380cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 0x7f05380cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33008/0 (socket says 172.21.15.50:33008) 2023-10-19T10:37:49.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.931+0000 7f053659c640 1 -- 172.21.15.50:0/1746163504 learned_addr learned my addr 172.21.15.50:0/1746163504 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:49.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f0536d9d640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0538069a70 0x7f0538151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053759e640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05380cb6f0 0x7f053814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053659c640 1 -- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0538069a70 msgr2=0x7f0538151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053659c640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0538069a70 0x7f0538151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053659c640 1 -- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05380cb6f0 msgr2=0x7f053814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:49.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053659c640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05380cb6f0 0x7f053814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:49.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053659c640 1 -- 172.21.15.50:0/1746163504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300094d0 con 0x7f0538069e50 2023-10-19T10:37:49.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053759e640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05380cb6f0 0x7f053814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:49.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053659c640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 0x7f05380cd760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f052c00d5c0 tx=0x7f052c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0125f0 con 0x7f0538069e50 2023-10-19T10:37:49.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c012750 con 0x7f0538069e50 2023-10-19T10:37:49.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05381f59a0 con 0x7f0538069e50 2023-10-19T10:37:49.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c012b00 con 0x7f0538069e50 2023-10-19T10:37:49.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.932+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05381f5f60 con 0x7f0538069e50 2023-10-19T10:37:49.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.933+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053804e420 con 0x7f0538069e50 2023-10-19T10:37:49.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.934+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f052c012c60 con 0x7f0538069e50 2023-10-19T10:37:49.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.934+0000 7f051f7fe640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f052406a050 0x7f052406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:49.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.934+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f052c088c20 con 0x7f0538069e50 2023-10-19T10:37:49.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.934+0000 7f0536d9d640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f052406a050 0x7f052406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:49.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.935+0000 7f0536d9d640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f052406a050 0x7f052406c510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0530036000 tx=0x7f053003d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:49.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:49.938+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f052c054070 con 0x7f0538069e50 2023-10-19T10:37:50.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.114+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f05380afe10 con 0x7f0538069e50 2023-10-19T10:37:50.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.114+0000 7f051f7fe640 1 -- 172.21.15.50:0/1746163504 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f052c057690 con 0x7f0538069e50 2023-10-19T10:37:50.117 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:50.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f052406a050 msgr2=0x7f052406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f052406a050 0x7f052406c510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0530036000 tx=0x7f053003d040 comp rx=0 tx=0).stop 2023-10-19T10:37:50.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 msgr2=0x7f05380cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 0x7f05380cd760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f052c00d5c0 tx=0x7f052c00da80 comp rx=0 tx=0).stop 2023-10-19T10:37:50.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 shutdown_connections 2023-10-19T10:37:50.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f052406a050 0x7f052406c510 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05380cb6f0 0x7f053814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0538069e50 0x7f05380cd760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 --2- 172.21.15.50:0/1746163504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0538069a70 0x7f0538151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 >> 172.21.15.50:0/1746163504 conn(0x7f05380cbc00 msgr2=0x7f053815c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 shutdown_connections 2023-10-19T10:37:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.116+0000 7f053d078640 1 -- 172.21.15.50:0/1746163504 wait complete. 2023-10-19T10:37:50.130 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-19T10:37:50.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.236+0000 7facef999640 1 Processor -- start 2023-10-19T10:37:50.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facef999640 1 -- start start 2023-10-19T10:37:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facef999640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face815c510 0x7face80cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facef999640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7face80b10b0 0x7face8160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facef999640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80b7ca0 con 0x7face80d23b0 2023-10-19T10:37:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facef999640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face80d21c0 con 0x7face80b10b0 2023-10-19T10:37:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facef999640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80b7ef0 con 0x7face815c510 2023-10-19T10:37:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facecf34640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7face80b10b0 0x7face8160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facedf36640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7face80d23b0 0x7face80d06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44702/0 (socket says 172.21.15.50:44702) 2023-10-19T10:37:50.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.237+0000 7facedf36640 1 -- 172.21.15.50:0/811993370 learned_addr learned my addr 172.21.15.50:0/811993370 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:50.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.238+0000 7faced735640 1 --2- 172.21.15.50:0/811993370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face815c510 0x7face80cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.238+0000 7facecf34640 1 -- 172.21.15.50:0/811993370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face815c510 msgr2=0x7face80cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:50.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.238+0000 7facecf34640 1 --2- 172.21.15.50:0/811993370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face815c510 0x7face80cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.238+0000 7facecf34640 1 -- 172.21.15.50:0/811993370 >> v1:172.21.15.50:6789/0 conn(0x7face80d23b0 legacy=0x7face80d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.238+0000 7face7fff640 1 -- 172.21.15.50:0/811993370 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051114134 0 0) 0x7face80b7ca0 con 0x7face80d23b0 2023-10-19T10:37:50.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.238+0000 7facecf34640 1 -- 172.21.15.50:0/811993370 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80b81d0 con 0x7face80b10b0 2023-10-19T10:37:50.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facecf34640 1 --2- 172.21.15.50:0/811993370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face80b10b0 0x7face8160bd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b77746d111dd985 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:50.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7face7fff640 1 -- 172.21.15.50:0/811993370 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc031830 con 0x7face80b10b0 2023-10-19T10:37:50.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7face7fff640 1 -- 172.21.15.50:0/811993370 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc031990 con 0x7face80b10b0 2023-10-19T10:37:50.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7face7fff640 1 -- 172.21.15.50:0/811993370 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc031d20 con 0x7face80b10b0 2023-10-19T10:37:50.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 -- 172.21.15.50:0/811993370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face80b10b0 msgr2=0x7face8160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 --2- 172.21.15.50:0/811993370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face80b10b0 0x7face8160bd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 -- 172.21.15.50:0/811993370 shutdown_connections 2023-10-19T10:37:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 --2- 172.21.15.50:0/811993370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face80b10b0 0x7face8160bd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 --2- 172.21.15.50:0/811993370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face815c510 0x7face80cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 -- 172.21.15.50:0/811993370 >> 172.21.15.50:0/811993370 conn(0x7face8071320 msgr2=0x7face8071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 -- 172.21.15.50:0/811993370 shutdown_connections 2023-10-19T10:37:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.239+0000 7facef999640 1 -- 172.21.15.50:0/811993370 wait complete. 2023-10-19T10:37:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 Processor -- start 2023-10-19T10:37:50.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 -- start start 2023-10-19T10:37:50.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face80b10b0 0x7face80c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face80d23b0 0x7face815d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 0x7face8156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80ca800 con 0x7face815c510 2023-10-19T10:37:50.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face815dad0 con 0x7face80b10b0 2023-10-19T10:37:50.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facef999640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80c9bf0 con 0x7face80d23b0 2023-10-19T10:37:50.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facedf36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 0x7face8156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facedf36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 0x7face8156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33024/0 (socket says 172.21.15.50:33024) 2023-10-19T10:37:50.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.241+0000 7facedf36640 1 -- 172.21.15.50:0/1864147185 learned_addr learned my addr 172.21.15.50:0/1864147185 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7faced735640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face80b10b0 0x7face80c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facecf34640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face80d23b0 0x7face815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facedf36640 1 -- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face80d23b0 msgr2=0x7face815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facedf36640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face80d23b0 0x7face815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facedf36640 1 -- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face80b10b0 msgr2=0x7face80c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facedf36640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face80b10b0 0x7face80c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facedf36640 1 -- 172.21.15.50:0/1864147185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009400 con 0x7face815c510 2023-10-19T10:37:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7faced735640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face80b10b0 0x7face80c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:50.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facedf36640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 0x7face8156fe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facd800c7e0 tx=0x7facd800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:50.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800a640 con 0x7face815c510 2023-10-19T10:37:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800a7c0 con 0x7face815c510 2023-10-19T10:37:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face80c9ed0 con 0x7face815c510 2023-10-19T10:37:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800ab70 con 0x7face815c510 2023-10-19T10:37:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.242+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face80ca490 con 0x7face815c510 2023-10-19T10:37:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.243+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face80ca970 con 0x7face815c510 2023-10-19T10:37:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.244+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7facd800acd0 con 0x7face815c510 2023-10-19T10:37:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.244+0000 7face5ffb640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7facd406a050 0x7facd406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.244+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7facd80881f0 con 0x7face815c510 2023-10-19T10:37:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.245+0000 7faced735640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7facd406a050 0x7facd406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.245+0000 7faced735640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7facd406a050 0x7facd406c510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7face0009b90 tx=0x7face0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:50.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.249+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7facd8053510 con 0x7face815c510 2023-10-19T10:37:50.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.429+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7face804e420 con 0x7face815c510 2023-10-19T10:37:50.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.433+0000 7face5ffb640 1 -- 172.21.15.50:0/1864147185 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7facd8056b30 con 0x7face815c510 2023-10-19T10:37:50.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7facd406a050 msgr2=0x7facd406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7facd406a050 0x7facd406c510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7face0009b90 tx=0x7face0008040 comp rx=0 tx=0).stop 2023-10-19T10:37:50.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 msgr2=0x7face8156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 0x7face8156fe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facd800c7e0 tx=0x7facd800cca0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 shutdown_connections 2023-10-19T10:37:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7facd406a050 0x7facd406c510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7face815c510 0x7face8156fe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7face80d23b0 0x7face815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 --2- 172.21.15.50:0/1864147185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face80b10b0 0x7face80c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 >> 172.21.15.50:0/1864147185 conn(0x7face8071320 msgr2=0x7face80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:50.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 shutdown_connections 2023-10-19T10:37:50.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.435+0000 7facef999640 1 -- 172.21.15.50:0/1864147185 wait complete. 2023-10-19T10:37:50.449 INFO:tasks.cephfs_test_runner:Simulate software freeze or temporary network failure. ... skipped 'Require first client to be on separate server from MDSs' 2023-10-19T10:37:50.449 INFO:tasks.cephfs_test_runner:Starting test: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) If any client gets unresponsive/it's session get idle due to lagginess 2023-10-19T10:37:50.450 INFO:tasks.cephfs_test_runner:test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-19T10:37:50.450 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set' 2023-10-19T10:37:50.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.555+0000 7ff13d327640 1 Processor -- start 2023-10-19T10:37:50.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.555+0000 7ff13d327640 1 -- start start 2023-10-19T10:37:50.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff13d327640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff1380d0e30 0x7ff13815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff13d327640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff1380cd920 0x7ff1380d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff13d327640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13806aa30 con 0x7ff13815c570 2023-10-19T10:37:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff13d327640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13815ce90 con 0x7ff1380d0e30 2023-10-19T10:37:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff13d327640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13806b2d0 con 0x7ff1380cd920 2023-10-19T10:37:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff1380d0e30 0x7ff13815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff1380d0e30 0x7ff13815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33032/0 (socket says 172.21.15.50:33032) 2023-10-19T10:37:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 -- 172.21.15.50:0/2807998532 learned_addr learned my addr 172.21.15.50:0/2807998532 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1377fe640 1 --2- 172.21.15.50:0/2807998532 >> v2:172.21.15.183:3300/0 conn(0x7ff1380cd920 0x7ff1380d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 -- 172.21.15.50:0/2807998532 >> v2:172.21.15.183:3300/0 conn(0x7ff1380cd920 msgr2=0x7ff1380d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 --2- 172.21.15.50:0/2807998532 >> v2:172.21.15.183:3300/0 conn(0x7ff1380cd920 0x7ff1380d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 -- 172.21.15.50:0/2807998532 >> v1:172.21.15.183:6789/0 conn(0x7ff13815c570 legacy=0x7ff13806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.556+0000 7ff1367fc640 1 -- 172.21.15.50:0/2807998532 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1380d1270 con 0x7ff1380d0e30 2023-10-19T10:37:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.557+0000 7ff1367fc640 1 --2- 172.21.15.50:0/2807998532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380d0e30 0x7ff13815c950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1380d3770 tx=0x7ff12c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557e8948f6d56865 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.557+0000 7ff135ffb640 1 -- 172.21.15.50:0/2807998532 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0317e0 con 0x7ff1380d0e30 2023-10-19T10:37:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.557+0000 7ff135ffb640 1 -- 172.21.15.50:0/2807998532 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c031940 con 0x7ff1380d0e30 2023-10-19T10:37:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.557+0000 7ff135ffb640 1 -- 172.21.15.50:0/2807998532 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c031cd0 con 0x7ff1380d0e30 2023-10-19T10:37:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 -- 172.21.15.50:0/2807998532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380d0e30 msgr2=0x7ff13815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 --2- 172.21.15.50:0/2807998532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380d0e30 0x7ff13815c950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1380d3770 tx=0x7ff12c02b880 comp rx=0 tx=0).stop 2023-10-19T10:37:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 -- 172.21.15.50:0/2807998532 shutdown_connections 2023-10-19T10:37:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 --2- 172.21.15.50:0/2807998532 >> v2:172.21.15.183:3300/0 conn(0x7ff1380cd920 0x7ff1380d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 --2- 172.21.15.50:0/2807998532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380d0e30 0x7ff13815c950 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 -- 172.21.15.50:0/2807998532 >> 172.21.15.50:0/2807998532 conn(0x7ff138089ff0 msgr2=0x7ff13809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 -- 172.21.15.50:0/2807998532 shutdown_connections 2023-10-19T10:37:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.558+0000 7ff13d327640 1 -- 172.21.15.50:0/2807998532 wait complete. 2023-10-19T10:37:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.559+0000 7ff13d327640 1 Processor -- start 2023-10-19T10:37:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.559+0000 7ff13d327640 1 -- start start 2023-10-19T10:37:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff13d327640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 0x7ff13815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff13d327640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1380d0e30 0x7ff1380b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff13d327640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff13815c570 0x7ff1380b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff13d327640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1380c7ca0 con 0x7ff1380cd920 2023-10-19T10:37:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff13d327640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1380c7f80 con 0x7ff1380d0e30 2023-10-19T10:37:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff13d327640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1380c8260 con 0x7ff13815c570 2023-10-19T10:37:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 0x7ff13815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 0x7ff13815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33046/0 (socket says 172.21.15.50:33046) 2023-10-19T10:37:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 -- 172.21.15.50:0/2112758145 learned_addr learned my addr 172.21.15.50:0/2112758145 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff1377fe640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff13815c570 0x7ff1380b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff1367fc640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1380d0e30 0x7ff1380b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 -- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff13815c570 msgr2=0x7ff1380b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff13815c570 0x7ff1380b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 -- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1380d0e30 msgr2=0x7ff1380b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1380d0e30 0x7ff1380b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.560+0000 7ff136ffd640 1 -- 172.21.15.50:0/2112758145 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c0094d0 con 0x7ff1380cd920 2023-10-19T10:37:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.561+0000 7ff136ffd640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 0x7ff13815b640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff128004640 tx=0x7ff12800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.561+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128011790 con 0x7ff1380cd920 2023-10-19T10:37:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.561+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1280118f0 con 0x7ff1380cd920 2023-10-19T10:37:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.561+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1380c8540 con 0x7ff1380cd920 2023-10-19T10:37:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.561+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128011ce0 con 0x7ff1380cd920 2023-10-19T10:37:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.561+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1380b7240 con 0x7ff1380cd920 2023-10-19T10:37:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.562+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff138073990 con 0x7ff1380cd920 2023-10-19T10:37:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.562+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff128012420 con 0x7ff1380cd920 2023-10-19T10:37:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.562+0000 7ff11f7fe640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff12406a080 0x7ff12406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.563+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff1280885b0 con 0x7ff1380cd920 2023-10-19T10:37:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.563+0000 7ff1367fc640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff12406a080 0x7ff12406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.563+0000 7ff1367fc640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff12406a080 0x7ff12406c540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff12c036000 tx=0x7ff12c03d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.567+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff128053960 con 0x7ff1380cd920 2023-10-19T10:37:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:50.812+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]} v 0) v1 -- 0x7ff1380d3480 con 0x7ff1380cd920 2023-10-19T10:37:51.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.438+0000 7ff11f7fe640 1 -- 172.21.15.50:0/2112758145 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]}]=0 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff128056f80 con 0x7ff1380cd920 2023-10-19T10:37:51.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.440+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff12406a080 msgr2=0x7ff12406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.440+0000 7ff13d327640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff12406a080 0x7ff12406c540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff12c036000 tx=0x7ff12c03d040 comp rx=0 tx=0).stop 2023-10-19T10:37:51.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.440+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 msgr2=0x7ff13815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.440+0000 7ff13d327640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 0x7ff13815b640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff128004640 tx=0x7ff12800c8e0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 shutdown_connections 2023-10-19T10:37:51.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff12406a080 0x7ff12406c540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff13815c570 0x7ff1380b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1380d0e30 0x7ff1380b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 --2- 172.21.15.50:0/2112758145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff1380cd920 0x7ff13815b640 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 >> 172.21.15.50:0/2112758145 conn(0x7ff138089ff0 msgr2=0x7ff13815bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:51.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 shutdown_connections 2023-10-19T10:37:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.441+0000 7ff13d327640 1 -- 172.21.15.50:0/2112758145 wait complete. 2023-10-19T10:37:51.457 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:51.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.567+0000 7f1937aee640 1 Processor -- start 2023-10-19T10:37:51.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.567+0000 7f1937aee640 1 -- start start 2023-10-19T10:37:51.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f1937aee640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f193015c570 0x7f19300d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f1937aee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19300cd920 0x7f19300d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f1937aee640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193006aa30 con 0x7f193015c950 2023-10-19T10:37:51.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f1937aee640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193006b2d0 con 0x7f193015c570 2023-10-19T10:37:51.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f1937aee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19300ce090 con 0x7f19300cd920 2023-10-19T10:37:51.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f193015c570 0x7f19300d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f193015c570 0x7f19300d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33048/0 (socket says 172.21.15.50:33048) 2023-10-19T10:37:51.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 -- 172.21.15.50:0/2896725336 learned_addr learned my addr 172.21.15.50:0/2896725336 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:51.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193608b640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19300cd920 0x7f19300d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 -- 172.21.15.50:0/2896725336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19300cd920 msgr2=0x7f19300d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:51.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19300cd920 0x7f19300d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 -- 172.21.15.50:0/2896725336 >> v1:172.21.15.183:6789/0 conn(0x7f193015c950 legacy=0x7f193006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:51.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193588a640 1 -- 172.21.15.50:0/2896725336 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193006ac80 con 0x7f193015c570 2023-10-19T10:37:51.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.568+0000 7f193608b640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19300cd920 0x7f19300d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:51.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.569+0000 7f193588a640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c570 0x7f19300d0e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1928009850 tx=0x7f192802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851eb6c53269f809 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:51.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.569+0000 7f1934888640 1 -- 172.21.15.50:0/2896725336 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928004230 con 0x7f193015c570 2023-10-19T10:37:51.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.569+0000 7f1934888640 1 -- 172.21.15.50:0/2896725336 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1928004390 con 0x7f193015c570 2023-10-19T10:37:51.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.569+0000 7f1934888640 1 -- 172.21.15.50:0/2896725336 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928031830 con 0x7f193015c570 2023-10-19T10:37:51.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 -- 172.21.15.50:0/2896725336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c570 msgr2=0x7f19300d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c570 0x7f19300d0e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1928009850 tx=0x7f192802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 -- 172.21.15.50:0/2896725336 shutdown_connections 2023-10-19T10:37:51.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19300cd920 0x7f19300d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 --2- 172.21.15.50:0/2896725336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c570 0x7f19300d0e30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 -- 172.21.15.50:0/2896725336 >> 172.21.15.50:0/2896725336 conn(0x7f1930089ff0 msgr2=0x7f193009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:51.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 -- 172.21.15.50:0/2896725336 shutdown_connections 2023-10-19T10:37:51.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.570+0000 7f1937aee640 1 -- 172.21.15.50:0/2896725336 wait complete. 2023-10-19T10:37:51.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.571+0000 7f1937aee640 1 Processor -- start 2023-10-19T10:37:51.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.571+0000 7f1937aee640 1 -- start start 2023-10-19T10:37:51.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19300cd920 0x7f193015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f193015c570 0x7f19300c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 0x7f193014b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c8160 con 0x7f193015c950 2023-10-19T10:37:51.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c8440 con 0x7f19300cd920 2023-10-19T10:37:51.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c8720 con 0x7f193015c570 2023-10-19T10:37:51.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 0x7f193014b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 0x7f193014b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33054/0 (socket says 172.21.15.50:33054) 2023-10-19T10:37:51.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 -- 172.21.15.50:0/2885823601 learned_addr learned my addr 172.21.15.50:0/2885823601 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:51.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193588a640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19300cd920 0x7f193015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1935089640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f193015c570 0x7f19300c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 -- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f193015c570 msgr2=0x7f19300c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f193015c570 0x7f19300c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 -- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19300cd920 msgr2=0x7f193015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19300cd920 0x7f193015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 -- 172.21.15.50:0/2885823601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1928009460 con 0x7f193015c950 2023-10-19T10:37:51.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1935089640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f193015c570 0x7f19300c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:51.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f193608b640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 0x7f193014b920 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f192c00cca0 tx=0x7f192c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:51.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00f680 con 0x7f193015c950 2023-10-19T10:37:51.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00f800 con 0x7f193015c950 2023-10-19T10:37:51.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19300c8a00 con 0x7f193015c950 2023-10-19T10:37:51.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00fbb0 con 0x7f193015c950 2023-10-19T10:37:51.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.572+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193015fda0 con 0x7f193015c950 2023-10-19T10:37:51.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.573+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19300c8f20 con 0x7f193015c950 2023-10-19T10:37:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.574+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f192c00b040 con 0x7f193015c950 2023-10-19T10:37:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.574+0000 7f19267fc640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f191406a050 0x7f191406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.574+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f192c0891d0 con 0x7f193015c950 2023-10-19T10:37:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.574+0000 7f193588a640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f191406a050 0x7f191406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.575+0000 7f193588a640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f191406a050 0x7f191406c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1928035000 tx=0x7f1928008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.578+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f192c054620 con 0x7f193015c950 2023-10-19T10:37:51.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.755+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1930073990 con 0x7f193015c950 2023-10-19T10:37:51.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.756+0000 7f19267fc640 1 -- 172.21.15.50:0/2885823601 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f192c057c40 con 0x7f193015c950 2023-10-19T10:37:51.759 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:51.759 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:51.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f191406a050 msgr2=0x7f191406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f191406a050 0x7f191406c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1928035000 tx=0x7f1928008030 comp rx=0 tx=0).stop 2023-10-19T10:37:51.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 msgr2=0x7f193014b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 0x7f193014b920 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f192c00cca0 tx=0x7f192c00a650 comp rx=0 tx=0).stop 2023-10-19T10:37:51.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 shutdown_connections 2023-10-19T10:37:51.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f191406a050 0x7f191406c510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f193015c950 0x7f193014b920 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f193015c570 0x7f19300c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 --2- 172.21.15.50:0/2885823601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19300cd920 0x7f193015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 >> 172.21.15.50:0/2885823601 conn(0x7f1930089ff0 msgr2=0x7f19300b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:51.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 shutdown_connections 2023-10-19T10:37:51.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.758+0000 7f1937aee640 1 -- 172.21.15.50:0/2885823601 wait complete. 2023-10-19T10:37:51.771 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:51.772 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:51.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.879+0000 7fbd29b99640 1 Processor -- start 2023-10-19T10:37:51.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.879+0000 7fbd29b99640 1 -- start start 2023-10-19T10:37:51.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd29b99640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbd24151600 0x7fbd2415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd29b99640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2406c850 con 0x7fbd24151220 2023-10-19T10:37:51.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd29b99640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2406bfc0 con 0x7fbd2415e4b0 2023-10-19T10:37:51.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd29b99640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2415e940 con 0x7fbd24151600 2023-10-19T10:37:51.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd237fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fbd24151220 0x7fbd2406b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44716/0 (socket says 172.21.15.50:44716) 2023-10-19T10:37:51.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd237fe640 1 -- 172.21.15.50:0/1707361174 learned_addr learned my addr 172.21.15.50:0/1707361174 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:51.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd22ffd640 1 --2- 172.21.15.50:0/1707361174 >> v2:172.21.15.183:3300/0 conn(0x7fbd24151600 0x7fbd2415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017210398 0 0) 0x7fbd2406c850 con 0x7fbd24151220 2023-10-19T10:37:51.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd0c003520 con 0x7fbd24151220 2023-10-19T10:37:51.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 207369187 0 0) 0x7fbd0c003520 con 0x7fbd24151220 2023-10-19T10:37:51.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 >> v2:172.21.15.183:3300/0 conn(0x7fbd24151600 msgr2=0x7fbd2415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:51.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 --2- 172.21.15.50:0/1707361174 >> v2:172.21.15.183:3300/0 conn(0x7fbd24151600 0x7fbd2415df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 >> v1:172.21.15.183:6789/0 conn(0x7fbd2415e4b0 legacy=0x7fbd240ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2406c290 con 0x7fbd24151220 2023-10-19T10:37:51.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fbd14001670 con 0x7fbd24151220 2023-10-19T10:37:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 >> v1:172.21.15.50:6789/0 conn(0x7fbd24151220 legacy=0x7fbd2406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd0c003970 0x7fbd0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0047c0 0x7fbd0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd0c0052a0 0x7fbd0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2415e940 con 0x7fbd0c0052a0 2023-10-19T10:37:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00c170 con 0x7fbd0c003970 2023-10-19T10:37:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.880+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00c330 con 0x7fbd0c0047c0 2023-10-19T10:37:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd0c0052a0 0x7fbd0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd237fe640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd0c003970 0x7fbd0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd22ffd640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0047c0 0x7fbd0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 -- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0047c0 msgr2=0x7fbd0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:51.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0047c0 0x7fbd0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 -- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd0c003970 msgr2=0x7fbd0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:51.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd0c003970 0x7fbd0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 -- 172.21.15.50:0/1707361174 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c003520 con 0x7fbd0c0052a0 2023-10-19T10:37:51.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.881+0000 7fbd23fff640 1 -- 172.21.15.50:0/1707361174 reap_dead start 2023-10-19T10:37:51.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd0c0052a0 0x7fbd0c00b980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c001e20 tx=0x7fbd0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f308dd2b49c0fc81 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:51.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c003b70 con 0x7fbd0c0052a0 2023-10-19T10:37:51.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c003cd0 con 0x7fbd0c0052a0 2023-10-19T10:37:51.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd227fc640 1 -- 172.21.15.50:0/1707361174 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0052f0 con 0x7fbd0c0052a0 2023-10-19T10:37:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 -- 172.21.15.50:0/1707361174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd0c0052a0 msgr2=0x7fbd0c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd0c0052a0 0x7fbd0c00b980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c001e20 tx=0x7fbd0c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 -- 172.21.15.50:0/1707361174 shutdown_connections 2023-10-19T10:37:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1707361174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd0c0052a0 0x7fbd0c00b980 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 -- 172.21.15.50:0/1707361174 >> 172.21.15.50:0/1707361174 conn(0x7fbd240afbe0 msgr2=0x7fbd240affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:51.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 -- 172.21.15.50:0/1707361174 shutdown_connections 2023-10-19T10:37:51.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.882+0000 7fbd29b99640 1 -- 172.21.15.50:0/1707361174 wait complete. 2023-10-19T10:37:51.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.883+0000 7fbd29b99640 1 Processor -- start 2023-10-19T10:37:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 -- start start 2023-10-19T10:37:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0052a0 0x7fbd240acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd241f51f0 0x7fbd241f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 0x7fbd240b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241f5bb0 con 0x7fbd240b2d40 2023-10-19T10:37:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240b3700 con 0x7fbd241f51f0 2023-10-19T10:37:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd29b99640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240aba80 con 0x7fbd0c0052a0 2023-10-19T10:37:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.884+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 0x7fbd240b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 0x7fbd240b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33070/0 (socket says 172.21.15.50:33070) 2023-10-19T10:37:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 -- 172.21.15.50:0/1539196508 learned_addr learned my addr 172.21.15.50:0/1539196508 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd22ffd640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd241f51f0 0x7fbd241f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 -- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0052a0 msgr2=0x7fbd240acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd237fe640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0052a0 0x7fbd240acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0052a0 0x7fbd240acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 -- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd241f51f0 msgr2=0x7fbd241f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd241f51f0 0x7fbd241f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 -- 172.21.15.50:0/1539196508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240abd60 con 0x7fbd240b2d40 2023-10-19T10:37:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd22ffd640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd241f51f0 0x7fbd241f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd23fff640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 0x7fbd240b31c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c001f40 tx=0x7fbd1c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c039450 con 0x7fbd240b2d40 2023-10-19T10:37:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c03d070 con 0x7fbd240b2d40 2023-10-19T10:37:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd240ac040 con 0x7fbd240b2d40 2023-10-19T10:37:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0397f0 con 0x7fbd240b2d40 2023-10-19T10:37:51.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.885+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd240ac600 con 0x7fbd240b2d40 2023-10-19T10:37:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.886+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd24071410 con 0x7fbd240b2d40 2023-10-19T10:37:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.887+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbd1c03f030 con 0x7fbd240b2d40 2023-10-19T10:37:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.887+0000 7fbd0bfff640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbd1006a050 0x7fbd1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.887+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbd1c003e40 con 0x7fbd240b2d40 2023-10-19T10:37:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.887+0000 7fbd237fe640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbd1006a050 0x7fbd1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.888+0000 7fbd237fe640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbd1006a050 0x7fbd1006c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd240ad980 tx=0x7fbd14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:51.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:51.891+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbd1c074ce0 con 0x7fbd240b2d40 2023-10-19T10:37:52.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.069+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd2404e420 con 0x7fbd240b2d40 2023-10-19T10:37:52.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.069+0000 7fbd0bfff640 1 -- 172.21.15.50:0/1539196508 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd1c078300 con 0x7fbd240b2d40 2023-10-19T10:37:52.072 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:52.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbd1006a050 msgr2=0x7fbd1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbd1006a050 0x7fbd1006c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd240ad980 tx=0x7fbd14009040 comp rx=0 tx=0).stop 2023-10-19T10:37:52.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 msgr2=0x7fbd240b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 0x7fbd240b31c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c001f40 tx=0x7fbd1c0014a0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 shutdown_connections 2023-10-19T10:37:52.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbd1006a050 0x7fbd1006c510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd240b2d40 0x7fbd240b31c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd241f51f0 0x7fbd241f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 --2- 172.21.15.50:0/1539196508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbd0c0052a0 0x7fbd240acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 >> 172.21.15.50:0/1539196508 conn(0x7fbd240afbe0 msgr2=0x7fbd240aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 shutdown_connections 2023-10-19T10:37:52.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.071+0000 7fbd29b99640 1 -- 172.21.15.50:0/1539196508 wait complete. 2023-10-19T10:37:52.085 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:52.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.192+0000 7f5fa2111640 1 Processor -- start 2023-10-19T10:37:52.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.192+0000 7f5fa2111640 1 -- start start 2023-10-19T10:37:52.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5fa2111640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 0x7f5f9c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5fa2111640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 0x7f5f9c160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5fa2111640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0af250 con 0x7f5f9c0b3cd0 2023-10-19T10:37:52.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5fa2111640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b0190 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5fa2111640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0af510 con 0x7f5f9c0b3090 2023-10-19T10:37:52.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 0x7f5f9c160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9b7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 0x7f5f9c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9bfff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f5f9c0b3cd0 0x7f5f9c0b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:59990/0 (socket says 172.21.15.50:59990) 2023-10-19T10:37:52.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 0x7f5f9c160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:60714/0 (socket says 172.21.15.50:60714) 2023-10-19T10:37:52.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9b7fe640 1 --2- 172.21.15.50:0/3243320543 >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 0x7f5f9c0d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:44442/0 (socket says 172.21.15.50:44442) 2023-10-19T10:37:52.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9bfff640 1 -- 172.21.15.50:0/3243320543 learned_addr learned my addr 172.21.15.50:0/3243320543 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:52.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.193+0000 7f5f9a7fc640 1 -- 172.21.15.50:0/3243320543 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792930198 0 0) 0x7f5f9c0af250 con 0x7f5f9c0b3cd0 2023-10-19T10:37:52.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9a7fc640 1 -- 172.21.15.50:0/3243320543 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f84003520 con 0x7f5f9c0b3cd0 2023-10-19T10:37:52.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9affd640 1 -- 172.21.15.50:0/3243320543 >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 msgr2=0x7f5f9c0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:52.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9affd640 1 --2- 172.21.15.50:0/3243320543 >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 0x7f5f9c0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9affd640 1 -- 172.21.15.50:0/3243320543 >> v1:172.21.15.183:6789/0 conn(0x7f5f9c0b3cd0 legacy=0x7f5f9c0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9affd640 1 -- 172.21.15.50:0/3243320543 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0af7f0 con 0x7f5f9c0b3090 2023-10-19T10:37:52.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9b7fe640 1 --2- 172.21.15.50:0/3243320543 >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 0x7f5f9c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.194+0000 7f5f9affd640 1 --2- 172.21.15.50:0/3243320543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 0x7f5f9c160b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f900097a0 tx=0x7f5f9002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ce30530655ce94 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5f9a7fc640 1 -- 172.21.15.50:0/3243320543 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9002d040 con 0x7f5f9c0b3090 2023-10-19T10:37:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5f9a7fc640 1 -- 172.21.15.50:0/3243320543 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f90031810 con 0x7f5f9c0b3090 2023-10-19T10:37:52.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5f9a7fc640 1 -- 172.21.15.50:0/3243320543 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90031ba0 con 0x7f5f9c0b3090 2023-10-19T10:37:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 -- 172.21.15.50:0/3243320543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 msgr2=0x7f5f9c160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3243320543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 0x7f5f9c160b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f900097a0 tx=0x7f5f9002b8b0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 -- 172.21.15.50:0/3243320543 shutdown_connections 2023-10-19T10:37:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3243320543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3090 0x7f5f9c160b00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3243320543 >> v2:172.21.15.183:3300/0 conn(0x7f5f9c0b38f0 0x7f5f9c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 -- 172.21.15.50:0/3243320543 >> 172.21.15.50:0/3243320543 conn(0x7f5f9c09b960 msgr2=0x7f5f9c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 -- 172.21.15.50:0/3243320543 shutdown_connections 2023-10-19T10:37:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.195+0000 7f5fa2111640 1 -- 172.21.15.50:0/3243320543 wait complete. 2023-10-19T10:37:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.196+0000 7f5fa2111640 1 Processor -- start 2023-10-19T10:37:52.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.196+0000 7f5fa2111640 1 -- start start 2023-10-19T10:37:52.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5fa2111640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c0b3090 0x7f5f9c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5fa2111640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 0x7f5f9c0b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5fa2111640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3cd0 0x7f5f9c1e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5fa2111640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b7d10 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5fa2111640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b7f40 con 0x7f5f9c0b3090 2023-10-19T10:37:52.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5fa2111640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b64e0 con 0x7f5f9c0b3cd0 2023-10-19T10:37:52.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5f9affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 0x7f5f9c0b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5f9affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 0x7f5f9c0b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33076/0 (socket says 172.21.15.50:33076) 2023-10-19T10:37:52.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5f9affd640 1 -- 172.21.15.50:0/3454161215 learned_addr learned my addr 172.21.15.50:0/3454161215 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:52.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5f9b7fe640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c0b3090 0x7f5f9c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.197+0000 7f5f9bfff640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3cd0 0x7f5f9c1e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f9affd640 1 -- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3cd0 msgr2=0x7f5f9c1e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f9affd640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3cd0 0x7f5f9c1e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f9affd640 1 -- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c0b3090 msgr2=0x7f5f9c14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f9affd640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c0b3090 0x7f5f9c14bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f9affd640 1 -- 172.21.15.50:0/3454161215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0093a0 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f9affd640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 0x7f5f9c0b77d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f900098c0 tx=0x7f5f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9003ad60 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f90044400 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900445b0 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f90009400 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.198+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c0b6aa0 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.199+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c0b6c30 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.200+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5f9004d440 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.200+0000 7f5f83fff640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8806a050 0x7f5f8806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.200+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5f900aac30 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.200+0000 7f5f9b7fe640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8806a050 0x7f5f8806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.201+0000 7f5f9b7fe640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8806a050 0x7f5f8806c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009cc0 tx=0x7f5f8c009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.204+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5f900761c0 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.380+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f9c04e420 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.381+0000 7f5f83fff640 1 -- 172.21.15.50:0/3454161215 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f90035020 con 0x7f5f9c0b38f0 2023-10-19T10:37:52.384 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8806a050 msgr2=0x7f5f8806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8806a050 0x7f5f8806c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009cc0 tx=0x7f5f8c009110 comp rx=0 tx=0).stop 2023-10-19T10:37:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 msgr2=0x7f5f9c0b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 0x7f5f9c0b77d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f900098c0 tx=0x7f5f900046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 shutdown_connections 2023-10-19T10:37:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8806a050 0x7f5f8806c510 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5f9c0b3cd0 0x7f5f9c1e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c0b38f0 0x7f5f9c0b77d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 --2- 172.21.15.50:0/3454161215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c0b3090 0x7f5f9c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 >> 172.21.15.50:0/3454161215 conn(0x7f5f9c09b960 msgr2=0x7f5f9c0b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 shutdown_connections 2023-10-19T10:37:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.383+0000 7f5fa2111640 1 -- 172.21.15.50:0/3454161215 wait complete. 2023-10-19T10:37:52.396 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:52.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.503+0000 7f50ea343640 1 Processor -- start 2023-10-19T10:37:52.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.503+0000 7f50ea343640 1 -- start start 2023-10-19T10:37:52.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50ea343640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e4159f80 0x7f50e40caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50ea343640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f50e40caf40 0x7f50e40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50ea343640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f50e40cc170 0x7f50e4141d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50ea343640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e41600d0 con 0x7f50e40cc170 2023-10-19T10:37:52.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50ea343640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e40cb380 con 0x7f50e40caf40 2023-10-19T10:37:52.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50ea343640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4160320 con 0x7f50e4159f80 2023-10-19T10:37:52.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50e88e0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f50e40cc170 0x7f50e4141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50e88e0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f50e40cc170 0x7f50e4141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33078/0 (socket says 172.21.15.50:33078) 2023-10-19T10:37:52.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50e88e0640 1 -- 172.21.15.50:0/3487136369 learned_addr learned my addr 172.21.15.50:0/3487136369 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50e3fff640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e4159f80 0x7f50e40caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50e37fe640 1 --2- 172.21.15.50:0/3487136369 >> v2:172.21.15.183:3300/0 conn(0x7f50e40caf40 0x7f50e40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.504+0000 7f50e88e0640 1 -- 172.21.15.50:0/3487136369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e4159f80 msgr2=0x7f50e40caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e88e0640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e4159f80 0x7f50e40caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e88e0640 1 -- 172.21.15.50:0/3487136369 >> v2:172.21.15.183:3300/0 conn(0x7f50e40caf40 msgr2=0x7f50e40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:52.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e88e0640 1 --2- 172.21.15.50:0/3487136369 >> v2:172.21.15.183:3300/0 conn(0x7f50e40caf40 0x7f50e40cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e88e0640 1 -- 172.21.15.50:0/3487136369 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4160600 con 0x7f50e40cc170 2023-10-19T10:37:52.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e37fe640 1 --2- 172.21.15.50:0/3487136369 >> v2:172.21.15.183:3300/0 conn(0x7f50e40caf40 0x7f50e40cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:52.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e3fff640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e4159f80 0x7f50e40caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:52.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e88e0640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e40cc170 0x7f50e4141d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50e40b0c80 tx=0x7f50dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd702cc7e95abd71 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.505+0000 7f50e2ffd640 1 -- 172.21.15.50:0/3487136369 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc02ca80 con 0x7f50e40cc170 2023-10-19T10:37:52.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50e2ffd640 1 -- 172.21.15.50:0/3487136369 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc02cbe0 con 0x7f50e40cc170 2023-10-19T10:37:52.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50e2ffd640 1 -- 172.21.15.50:0/3487136369 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0343f0 con 0x7f50e40cc170 2023-10-19T10:37:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 -- 172.21.15.50:0/3487136369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e40cc170 msgr2=0x7f50e4141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e40cc170 0x7f50e4141d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50e40b0c80 tx=0x7f50dc02bee0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 -- 172.21.15.50:0/3487136369 shutdown_connections 2023-10-19T10:37:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e40cc170 0x7f50e4141d10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 --2- 172.21.15.50:0/3487136369 >> v2:172.21.15.183:3300/0 conn(0x7f50e40caf40 0x7f50e40cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 --2- 172.21.15.50:0/3487136369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e4159f80 0x7f50e40caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 -- 172.21.15.50:0/3487136369 >> 172.21.15.50:0/3487136369 conn(0x7f50e40afbe0 msgr2=0x7f50e40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 -- 172.21.15.50:0/3487136369 shutdown_connections 2023-10-19T10:37:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.506+0000 7f50ea343640 1 -- 172.21.15.50:0/3487136369 wait complete. 2023-10-19T10:37:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.507+0000 7f50ea343640 1 Processor -- start 2023-10-19T10:37:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 -- start start 2023-10-19T10:37:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e40caf40 0x7f50e41575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e40cc170 0x7f50e415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 0x7f50e415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4157af0 con 0x7f50e4159f80 2023-10-19T10:37:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e415b150 con 0x7f50e40caf40 2023-10-19T10:37:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50ea343640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e415b430 con 0x7f50e40cc170 2023-10-19T10:37:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50e88e0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 0x7f50e415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50e88e0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 0x7f50e415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33086/0 (socket says 172.21.15.50:33086) 2023-10-19T10:37:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.508+0000 7f50e88e0640 1 -- 172.21.15.50:0/3321251242 learned_addr learned my addr 172.21.15.50:0/3321251242 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e3fff640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e40caf40 0x7f50e41575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e37fe640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e40cc170 0x7f50e415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e88e0640 1 -- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e40cc170 msgr2=0x7f50e415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e88e0640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e40cc170 0x7f50e415e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e88e0640 1 -- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e40caf40 msgr2=0x7f50e41575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e88e0640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e40caf40 0x7f50e41575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e88e0640 1 -- 172.21.15.50:0/3321251242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009b30 con 0x7f50e4159f80 2023-10-19T10:37:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e88e0640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 0x7f50e415d6f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50dc036040 tx=0x7f50dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc009510 con 0x7f50e4159f80 2023-10-19T10:37:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc03e450 con 0x7f50e4159f80 2023-10-19T10:37:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e415b710 con 0x7f50e4159f80 2023-10-19T10:37:52.525 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc03e800 con 0x7f50e4159f80 2023-10-19T10:37:52.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.509+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e415bcd0 con 0x7f50e4159f80 2023-10-19T10:37:52.526 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.510+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e415bec0 con 0x7f50e4159f80 2023-10-19T10:37:52.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.511+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f50dc02cd50 con 0x7f50e4159f80 2023-10-19T10:37:52.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.511+0000 7f50e0ff9640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f50d006a050 0x7f50d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.511+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f50dc0ab470 con 0x7f50e4159f80 2023-10-19T10:37:52.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.511+0000 7f50e3fff640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f50d006a050 0x7f50d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.527 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.512+0000 7f50e3fff640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f50d006a050 0x7f50d006c510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50e40ccac0 tx=0x7f50d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.516+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f50dc0768e0 con 0x7f50e4159f80 2023-10-19T10:37:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.690+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50e404e420 con 0x7f50e4159f80 2023-10-19T10:37:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.690+0000 7f50e0ff9640 1 -- 172.21.15.50:0/3321251242 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f50dc079f00 con 0x7f50e4159f80 2023-10-19T10:37:52.693 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f50d006a050 msgr2=0x7f50d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f50d006a050 0x7f50d006c510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50e40ccac0 tx=0x7f50d8008040 comp rx=0 tx=0).stop 2023-10-19T10:37:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 msgr2=0x7f50e415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 0x7f50e415d6f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50dc036040 tx=0x7f50dc004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:52.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 shutdown_connections 2023-10-19T10:37:52.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f50d006a050 0x7f50d006c510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50e4159f80 0x7f50e415d6f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50e40cc170 0x7f50e415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 --2- 172.21.15.50:0/3321251242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e40caf40 0x7f50e41575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 >> 172.21.15.50:0/3321251242 conn(0x7f50e40afbe0 msgr2=0x7f50e4158550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:52.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 shutdown_connections 2023-10-19T10:37:52.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.692+0000 7f50ea343640 1 -- 172.21.15.50:0/3321251242 wait complete. 2023-10-19T10:37:52.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.812+0000 7fab303a7640 1 Processor -- start 2023-10-19T10:37:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.812+0000 7fab303a7640 1 -- start start 2023-10-19T10:37:52.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab303a7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab280d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab303a7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fab28161210 0x7fab280b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab303a7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab280af250 con 0x7fab280b38f0 2023-10-19T10:37:52.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab303a7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab280b0190 con 0x7fab28161210 2023-10-19T10:37:52.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab303a7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab280af510 con 0x7fab280b3090 2023-10-19T10:37:52.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e944640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fab28161210 0x7fab280b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e143640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fab280b38f0 0x7fab28160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44718/0 (socket says 172.21.15.50:44718) 2023-10-19T10:37:52.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e143640 1 -- 172.21.15.50:0/1257125012 learned_addr learned my addr 172.21.15.50:0/1257125012 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:52.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2d942640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab280d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e944640 1 -- 172.21.15.50:0/1257125012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 msgr2=0x7fab280d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:52.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e944640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab280d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e944640 1 -- 172.21.15.50:0/1257125012 >> v1:172.21.15.50:6789/0 conn(0x7fab280b38f0 legacy=0x7fab28160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2d141640 1 -- 172.21.15.50:0/1257125012 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965031910 0 0) 0x7fab280af250 con 0x7fab280b38f0 2023-10-19T10:37:52.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2e944640 1 -- 172.21.15.50:0/1257125012 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280af7f0 con 0x7fab28161210 2023-10-19T10:37:52.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.813+0000 7fab2d942640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab280d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:52.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.814+0000 7fab2e944640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab280b05b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fab200097a0 tx=0x7fab2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c280da7a290eed44 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.814+0000 7fab2d141640 1 -- 172.21.15.50:0/1257125012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20031830 con 0x7fab28161210 2023-10-19T10:37:52.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.814+0000 7fab2d141640 1 -- 172.21.15.50:0/1257125012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20031990 con 0x7fab28161210 2023-10-19T10:37:52.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.814+0000 7fab2d141640 1 -- 172.21.15.50:0/1257125012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20031d20 con 0x7fab28161210 2023-10-19T10:37:52.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.814+0000 7fab303a7640 1 -- 172.21.15.50:0/1257125012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 msgr2=0x7fab280b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.814+0000 7fab303a7640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab280b05b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fab200097a0 tx=0x7fab2002b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.815+0000 7fab303a7640 1 -- 172.21.15.50:0/1257125012 shutdown_connections 2023-10-19T10:37:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.815+0000 7fab303a7640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab280b05b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.815+0000 7fab303a7640 1 --2- 172.21.15.50:0/1257125012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab280d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.815+0000 7fab303a7640 1 -- 172.21.15.50:0/1257125012 >> 172.21.15.50:0/1257125012 conn(0x7fab2809b960 msgr2=0x7fab280d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:52.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.815+0000 7fab303a7640 1 -- 172.21.15.50:0/1257125012 shutdown_connections 2023-10-19T10:37:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.815+0000 7fab303a7640 1 -- 172.21.15.50:0/1257125012 wait complete. 2023-10-19T10:37:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.816+0000 7fab303a7640 1 Processor -- start 2023-10-19T10:37:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.816+0000 7fab303a7640 1 -- start start 2023-10-19T10:37:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.816+0000 7fab303a7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab2809fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.816+0000 7fab303a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab280b38f0 0x7fab280c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab303a7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab2815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab303a7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab281f5600 con 0x7fab28161210 2023-10-19T10:37:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab303a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab280caad0 con 0x7fab280b38f0 2023-10-19T10:37:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab303a7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab280cadb0 con 0x7fab280b3090 2023-10-19T10:37:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab2815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab2815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33096/0 (socket says 172.21.15.50:33096) 2023-10-19T10:37:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 -- 172.21.15.50:0/2096136658 learned_addr learned my addr 172.21.15.50:0/2096136658 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2d942640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab280b38f0 0x7fab280c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e143640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab2809fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 -- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 msgr2=0x7fab2809fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab2809fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 -- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab280b38f0 msgr2=0x7fab280c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab280b38f0 0x7fab280c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 -- 172.21.15.50:0/2096136658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab20009400 con 0x7fab28161210 2023-10-19T10:37:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab2e944640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab2815b0d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fab200097a0 tx=0x7fab200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2003a9e0 con 0x7fab28161210 2023-10-19T10:37:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2003ab40 con 0x7fab28161210 2023-10-19T10:37:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab280cb090 con 0x7fab28161210 2023-10-19T10:37:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab200443f0 con 0x7fab28161210 2023-10-19T10:37:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.817+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab280a01c0 con 0x7fab28161210 2023-10-19T10:37:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.818+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab281f5770 con 0x7fab28161210 2023-10-19T10:37:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.819+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fab2004d450 con 0x7fab28161210 2023-10-19T10:37:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.819+0000 7fab1effd640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fab0c06a080 0x7fab0c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.819+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fab200aa860 con 0x7fab28161210 2023-10-19T10:37:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.819+0000 7fab2e143640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fab0c06a080 0x7fab0c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.820+0000 7fab2e143640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fab0c06a080 0x7fab0c06c540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab24007fd0 tx=0x7fab24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:52.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.823+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fab20075e80 con 0x7fab28161210 2023-10-19T10:37:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.997+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab2804e420 con 0x7fab28161210 2023-10-19T10:37:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.997+0000 7fab1effd640 1 -- 172.21.15.50:0/2096136658 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab200794a0 con 0x7fab28161210 2023-10-19T10:37:53.000 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:53.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fab0c06a080 msgr2=0x7fab0c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fab0c06a080 0x7fab0c06c540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab24007fd0 tx=0x7fab24009040 comp rx=0 tx=0).stop 2023-10-19T10:37:53.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 msgr2=0x7fab2815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab2815b0d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fab200097a0 tx=0x7fab200046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 shutdown_connections 2023-10-19T10:37:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fab0c06a080 0x7fab0c06c540 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab28161210 0x7fab2815b0d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab280b38f0 0x7fab280c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 --2- 172.21.15.50:0/2096136658 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fab280b3090 0x7fab2809fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 >> 172.21.15.50:0/2096136658 conn(0x7fab2809b960 msgr2=0x7fab280d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:53.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 shutdown_connections 2023-10-19T10:37:53.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:52.999+0000 7fab303a7640 1 -- 172.21.15.50:0/2096136658 wait complete. 2023-10-19T10:37:53.012 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:53.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.119+0000 7faa0dc99640 1 Processor -- start 2023-10-19T10:37:53.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa0dc99640 1 -- start start 2023-10-19T10:37:53.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa0dc99640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faa08159300 0x7faa08071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa0dc99640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faa081596e0 0x7faa0815a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa0dc99640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0815a980 con 0x7faa080cdc00 2023-10-19T10:37:53.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa0dc99640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa080ce3c0 con 0x7faa081596e0 2023-10-19T10:37:53.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa0dc99640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa080ce6a0 con 0x7faa08159300 2023-10-19T10:37:53.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.120+0000 7faa06ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faa081596e0 0x7faa0815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa06ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faa081596e0 0x7faa0815a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33110/0 (socket says 172.21.15.50:33110) 2023-10-19T10:37:53.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa07fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7faa080cdc00 0x7faa08070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44734/0 (socket says 172.21.15.50:44734) 2023-10-19T10:37:53.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa06ffd640 1 -- 172.21.15.50:0/1794899377 learned_addr learned my addr 172.21.15.50:0/1794899377 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:53.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa077fe640 1 --2- 172.21.15.50:0/1794899377 >> v2:172.21.15.183:3300/0 conn(0x7faa08159300 0x7faa08071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa06ffd640 1 -- 172.21.15.50:0/1794899377 >> v2:172.21.15.183:3300/0 conn(0x7faa08159300 msgr2=0x7faa08071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:53.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa06ffd640 1 --2- 172.21.15.50:0/1794899377 >> v2:172.21.15.183:3300/0 conn(0x7faa08159300 0x7faa08071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa06ffd640 1 -- 172.21.15.50:0/1794899377 >> v1:172.21.15.50:6789/0 conn(0x7faa080cdc00 legacy=0x7faa08070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa06ffd640 1 -- 172.21.15.50:0/1794899377 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa08159c40 con 0x7faa081596e0 2023-10-19T10:37:53.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.121+0000 7faa067fc640 1 -- 172.21.15.50:0/1794899377 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126973258 0 0) 0x7faa00001490 con 0x7faa080cdc00 2023-10-19T10:37:53.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa06ffd640 1 --2- 172.21.15.50:0/1794899377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa081596e0 0x7faa0815a440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faa08070110 tx=0x7fa9fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303a9f33f06bdfc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa067fc640 1 -- 172.21.15.50:0/1794899377 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc030860 con 0x7faa081596e0 2023-10-19T10:37:53.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa067fc640 1 -- 172.21.15.50:0/1794899377 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc0309c0 con 0x7faa081596e0 2023-10-19T10:37:53.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa067fc640 1 -- 172.21.15.50:0/1794899377 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc030d50 con 0x7faa081596e0 2023-10-19T10:37:53.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 -- 172.21.15.50:0/1794899377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa081596e0 msgr2=0x7faa0815a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1794899377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa081596e0 0x7faa0815a440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faa08070110 tx=0x7fa9fc02b9a0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 -- 172.21.15.50:0/1794899377 shutdown_connections 2023-10-19T10:37:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1794899377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa081596e0 0x7faa0815a440 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1794899377 >> v2:172.21.15.183:3300/0 conn(0x7faa08159300 0x7faa08071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 -- 172.21.15.50:0/1794899377 >> 172.21.15.50:0/1794899377 conn(0x7faa080cbc00 msgr2=0x7faa080cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 -- 172.21.15.50:0/1794899377 shutdown_connections 2023-10-19T10:37:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.122+0000 7faa0dc99640 1 -- 172.21.15.50:0/1794899377 wait complete. 2023-10-19T10:37:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.123+0000 7faa0dc99640 1 Processor -- start 2023-10-19T10:37:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 -- start start 2023-10-19T10:37:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 0x7faa08154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08159300 0x7faa081ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa081596e0 0x7faa080c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b3d00 con 0x7faa080cdc00 2023-10-19T10:37:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b33d0 con 0x7faa08159300 2023-10-19T10:37:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa0dc99640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b36b0 con 0x7faa081596e0 2023-10-19T10:37:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.124+0000 7faa077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 0x7faa08154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 0x7faa08154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33126/0 (socket says 172.21.15.50:33126) 2023-10-19T10:37:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 -- 172.21.15.50:0/1094564246 learned_addr learned my addr 172.21.15.50:0/1094564246 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa06ffd640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08159300 0x7faa081ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa07fff640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa081596e0 0x7faa080c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 -- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa081596e0 msgr2=0x7faa080c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa081596e0 0x7faa080c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 -- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08159300 msgr2=0x7faa081ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08159300 0x7faa081ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 -- 172.21.15.50:0/1094564246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0094d0 con 0x7faa080cdc00 2023-10-19T10:37:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa07fff640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa081596e0 0x7faa080c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa077fe640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 0x7faa08154e20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800d800 tx=0x7fa9f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800b6c0 con 0x7faa080cdc00 2023-10-19T10:37:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800b820 con 0x7faa080cdc00 2023-10-19T10:37:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa080b3990 con 0x7faa080cdc00 2023-10-19T10:37:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800bbb0 con 0x7faa080cdc00 2023-10-19T10:37:53.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.125+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa080c7ae0 con 0x7faa080cdc00 2023-10-19T10:37:53.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.126+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0804e420 con 0x7faa080cdc00 2023-10-19T10:37:53.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.127+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9f800bd10 con 0x7faa080cdc00 2023-10-19T10:37:53.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.127+0000 7fa9effff640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9f406a050 0x7fa9f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.127+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa9f8088d10 con 0x7faa080cdc00 2023-10-19T10:37:53.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.127+0000 7faa06ffd640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9f406a050 0x7fa9f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.128+0000 7faa06ffd640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9f406a050 0x7fa9f406c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc036000 tx=0x7fa9fc03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.138 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.132+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa9f8054160 con 0x7faa080cdc00 2023-10-19T10:37:53.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.305+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faa080cbfe0 con 0x7faa080cdc00 2023-10-19T10:37:53.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.305+0000 7fa9effff640 1 -- 172.21.15.50:0/1094564246 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa9f8057780 con 0x7faa080cdc00 2023-10-19T10:37:53.308 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:53.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9f406a050 msgr2=0x7fa9f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9f406a050 0x7fa9f406c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc036000 tx=0x7fa9fc03b040 comp rx=0 tx=0).stop 2023-10-19T10:37:53.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 msgr2=0x7faa08154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 0x7faa08154e20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800d800 tx=0x7fa9f800dcc0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 shutdown_connections 2023-10-19T10:37:53.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9f406a050 0x7fa9f406c510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa081596e0 0x7faa080c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08159300 0x7faa081ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 --2- 172.21.15.50:0/1094564246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa080cdc00 0x7faa08154e20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 >> 172.21.15.50:0/1094564246 conn(0x7faa080cbc00 msgr2=0x7faa080afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:53.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 shutdown_connections 2023-10-19T10:37:53.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.307+0000 7faa0dc99640 1 -- 172.21.15.50:0/1094564246 wait complete. 2023-10-19T10:37:53.321 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-19T10:37:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.427+0000 7fcc78c7e640 1 Processor -- start 2023-10-19T10:37:53.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.427+0000 7fcc78c7e640 1 -- start start 2023-10-19T10:37:53.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc78c7e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b3090 0x7fcc740d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc78c7e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcc74161210 0x7fcc740b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc78c7e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc740af250 con 0x7fcc740b38f0 2023-10-19T10:37:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc78c7e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740b0190 con 0x7fcc74161210 2023-10-19T10:37:53.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc78c7e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740af510 con 0x7fcc740b3090 2023-10-19T10:37:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc72d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcc74161210 0x7fcc740b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc7259c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fcc740b38f0 0x7fcc74160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44738/0 (socket says 172.21.15.50:44738) 2023-10-19T10:37:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.428+0000 7fcc7259c640 1 -- 172.21.15.50:0/2137981496 learned_addr learned my addr 172.21.15.50:0/2137981496 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc7159a640 1 -- 172.21.15.50:0/2137981496 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704597725 0 0) 0x7fcc740af250 con 0x7fcc740b38f0 2023-10-19T10:37:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc71d9b640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b3090 0x7fcc740d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc7159a640 1 -- 172.21.15.50:0/2137981496 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc60003520 con 0x7fcc740b38f0 2023-10-19T10:37:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc72d9d640 1 -- 172.21.15.50:0/2137981496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b3090 msgr2=0x7fcc740d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc72d9d640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b3090 0x7fcc740d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc72d9d640 1 -- 172.21.15.50:0/2137981496 >> v1:172.21.15.50:6789/0 conn(0x7fcc740b38f0 legacy=0x7fcc74160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc72d9d640 1 -- 172.21.15.50:0/2137981496 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740af7f0 con 0x7fcc74161210 2023-10-19T10:37:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc71d9b640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b3090 0x7fcc740d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:53.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.429+0000 7fcc72d9d640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc740b05b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc6402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aadb4b5b9d1a5b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc7159a640 1 -- 172.21.15.50:0/2137981496 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6402cc50 con 0x7fcc74161210 2023-10-19T10:37:53.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc7159a640 1 -- 172.21.15.50:0/2137981496 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6402cdb0 con 0x7fcc74161210 2023-10-19T10:37:53.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc7159a640 1 -- 172.21.15.50:0/2137981496 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64034660 con 0x7fcc74161210 2023-10-19T10:37:53.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 -- 172.21.15.50:0/2137981496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 msgr2=0x7fcc740b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc740b05b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc6402bea0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 -- 172.21.15.50:0/2137981496 shutdown_connections 2023-10-19T10:37:53.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc740b05b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/2137981496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b3090 0x7fcc740d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 -- 172.21.15.50:0/2137981496 >> 172.21.15.50:0/2137981496 conn(0x7fcc7409b960 msgr2=0x7fcc740d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 -- 172.21.15.50:0/2137981496 shutdown_connections 2023-10-19T10:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.430+0000 7fcc78c7e640 1 -- 172.21.15.50:0/2137981496 wait complete. 2023-10-19T10:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 Processor -- start 2023-10-19T10:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 -- start start 2023-10-19T10:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc740b3090 0x7fcc741f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b38f0 0x7fcc7415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc7415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740cb3e0 con 0x7fcc74161210 2023-10-19T10:37:53.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740cae20 con 0x7fcc740b3090 2023-10-19T10:37:53.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc78c7e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740cb100 con 0x7fcc740b38f0 2023-10-19T10:37:53.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc72d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc7415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc72d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc7415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33146/0 (socket says 172.21.15.50:33146) 2023-10-19T10:37:53.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.432+0000 7fcc72d9d640 1 -- 172.21.15.50:0/1858181481 learned_addr learned my addr 172.21.15.50:0/1858181481 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:53.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc7259c640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc740b3090 0x7fcc741f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc71d9b640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b38f0 0x7fcc7415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc72d9d640 1 -- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b38f0 msgr2=0x7fcc7415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc72d9d640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b38f0 0x7fcc7415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc72d9d640 1 -- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc740b3090 msgr2=0x7fcc741f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc72d9d640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc740b3090 0x7fcc741f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc72d9d640 1 -- 172.21.15.50:0/1858181481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64009a70 con 0x7fcc74161210 2023-10-19T10:37:53.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc72d9d640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc7415d570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64040650 con 0x7fcc74161210 2023-10-19T10:37:53.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc640407b0 con 0x7fcc74161210 2023-10-19T10:37:53.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64040b60 con 0x7fcc74161210 2023-10-19T10:37:53.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc741f5710 con 0x7fcc74161210 2023-10-19T10:37:53.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.433+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc741f6430 con 0x7fcc74161210 2023-10-19T10:37:53.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.434+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc741f5570 con 0x7fcc74161210 2023-10-19T10:37:53.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.435+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcc64040cc0 con 0x7fcc74161210 2023-10-19T10:37:53.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.435+0000 7fcc6affd640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcc5406a050 0x7fcc5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.435+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcc640ab690 con 0x7fcc74161210 2023-10-19T10:37:53.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.435+0000 7fcc7259c640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcc5406a050 0x7fcc5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.436+0000 7fcc7259c640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcc5406a050 0x7fcc5406c510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0010e0 tx=0x7fcc6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.439+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fcc741f5570 con 0x7fcc74161210 2023-10-19T10:37:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.614+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcc741f5570 con 0x7fcc74161210 2023-10-19T10:37:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.614+0000 7fcc6affd640 1 -- 172.21.15.50:0/1858181481 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc741f5570 con 0x7fcc74161210 2023-10-19T10:37:53.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcc5406a050 msgr2=0x7fcc5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcc5406a050 0x7fcc5406c510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0010e0 tx=0x7fcc6c009040 comp rx=0 tx=0).stop 2023-10-19T10:37:53.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 msgr2=0x7fcc7415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc7415d570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009d90 tx=0x7fcc64004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:53.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 shutdown_connections 2023-10-19T10:37:53.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcc5406a050 0x7fcc5406c510 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc74161210 0x7fcc7415d570 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc740b38f0 0x7fcc7415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 --2- 172.21.15.50:0/1858181481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc740b3090 0x7fcc741f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 >> 172.21.15.50:0/1858181481 conn(0x7fcc7409b960 msgr2=0x7fcc740d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:53.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 shutdown_connections 2023-10-19T10:37:53.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.616+0000 7fcc78c7e640 1 -- 172.21.15.50:0/1858181481 wait complete. 2023-10-19T10:37:53.629 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-19T10:37:53.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-19T10:37:53.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-19T10:37:53.730 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-19T10:37:53.731 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:37:53.731 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:53.747 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:37:53.748 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:53.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:53.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.946+0000 7f392937f640 1 Processor -- start 2023-10-19T10:37:53.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.946+0000 7f392937f640 1 -- start start 2023-10-19T10:37:53.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f392937f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39240d0e30 0x7f392415c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f392937f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f39240cd920 0x7f39240d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f392937f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392406aa30 con 0x7f392415c570 2023-10-19T10:37:53.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f392937f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392415ce90 con 0x7f39240d0e30 2023-10-19T10:37:53.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f392937f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392406b2d0 con 0x7f39240cd920 2023-10-19T10:37:53.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39240d0e30 0x7f392415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39240d0e30 0x7f392415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33148/0 (socket says 172.21.15.50:33148) 2023-10-19T10:37:53.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 -- 172.21.15.50:0/3177720633 learned_addr learned my addr 172.21.15.50:0/3177720633 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:53.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 -- 172.21.15.50:0/3177720633 >> v2:172.21.15.183:3300/0 conn(0x7f39240cd920 msgr2=0x7f39240d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:53.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39237fe640 1 --2- 172.21.15.50:0/3177720633 >> v2:172.21.15.183:3300/0 conn(0x7f39240cd920 0x7f39240d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 --2- 172.21.15.50:0/3177720633 >> v2:172.21.15.183:3300/0 conn(0x7f39240cd920 0x7f39240d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 -- 172.21.15.50:0/3177720633 >> v1:172.21.15.50:6789/0 conn(0x7f392415c570 legacy=0x7f392406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.947+0000 7f39227fc640 1 -- 172.21.15.50:0/3177720633 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240d1270 con 0x7f39240d0e30 2023-10-19T10:37:53.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.948+0000 7f39227fc640 1 --2- 172.21.15.50:0/3177720633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f392415c950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f39240d3770 tx=0x7f390c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c0f357f050a1a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.948+0000 7f3921ffb640 1 -- 172.21.15.50:0/3177720633 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c031870 con 0x7f39240d0e30 2023-10-19T10:37:53.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.948+0000 7f3921ffb640 1 -- 172.21.15.50:0/3177720633 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c0319d0 con 0x7f39240d0e30 2023-10-19T10:37:53.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.948+0000 7f3921ffb640 1 -- 172.21.15.50:0/3177720633 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c031d60 con 0x7f39240d0e30 2023-10-19T10:37:53.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 -- 172.21.15.50:0/3177720633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 msgr2=0x7f392415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 --2- 172.21.15.50:0/3177720633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f392415c950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f39240d3770 tx=0x7f390c02b910 comp rx=0 tx=0).stop 2023-10-19T10:37:53.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 -- 172.21.15.50:0/3177720633 shutdown_connections 2023-10-19T10:37:53.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 --2- 172.21.15.50:0/3177720633 >> v2:172.21.15.183:3300/0 conn(0x7f39240cd920 0x7f39240d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 --2- 172.21.15.50:0/3177720633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f392415c950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 -- 172.21.15.50:0/3177720633 >> 172.21.15.50:0/3177720633 conn(0x7f3924089ff0 msgr2=0x7f392409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:53.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 -- 172.21.15.50:0/3177720633 shutdown_connections 2023-10-19T10:37:53.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.949+0000 7f392937f640 1 -- 172.21.15.50:0/3177720633 wait complete. 2023-10-19T10:37:53.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.950+0000 7f392937f640 1 Processor -- start 2023-10-19T10:37:53.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.950+0000 7f392937f640 1 -- start start 2023-10-19T10:37:53.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f392937f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39240cd920 0x7f39240cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f392937f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f39240b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f392937f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f392415c570 0x7f39240c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f392937f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240ca2b0 con 0x7f39240d0e30 2023-10-19T10:37:53.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f392937f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240ca590 con 0x7f39240cd920 2023-10-19T10:37:53.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f392937f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240ca870 con 0x7f392415c570 2023-10-19T10:37:53.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f39240b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f39240b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33160/0 (socket says 172.21.15.50:33160) 2023-10-19T10:37:53.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 -- 172.21.15.50:0/2598751575 learned_addr learned my addr 172.21.15.50:0/2598751575 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:53.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f3922ffd640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39240cd920 0x7f39240cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39237fe640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f392415c570 0x7f39240c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 -- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f392415c570 msgr2=0x7f39240c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f392415c570 0x7f39240c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 -- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39240cd920 msgr2=0x7f39240cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:53.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39240cd920 0x7f39240cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:53.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 -- 172.21.15.50:0/2598751575 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0094d0 con 0x7f39240d0e30 2023-10-19T10:37:53.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f39227fc640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f39240b5cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f390c02b950 tx=0x7f390c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c031820 con 0x7f39240d0e30 2023-10-19T10:37:53.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.951+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c0443f0 con 0x7f39240d0e30 2023-10-19T10:37:53.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.952+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0447a0 con 0x7f39240d0e30 2023-10-19T10:37:53.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.952+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39240cab50 con 0x7f39240d0e30 2023-10-19T10:37:53.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.952+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39240cb110 con 0x7f39240d0e30 2023-10-19T10:37:53.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.953+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39240cb300 con 0x7f39240d0e30 2023-10-19T10:37:53.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.953+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f390c04d450 con 0x7f39240d0e30 2023-10-19T10:37:53.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.953+0000 7f391b7fe640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f390406a050 0x7f390406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:53.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.953+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f390c02d040 con 0x7f39240d0e30 2023-10-19T10:37:53.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.954+0000 7f3922ffd640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f390406a050 0x7f390406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:53.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.954+0000 7f3922ffd640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f390406a050 0x7f390406c510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3914007fd0 tx=0x7f3914009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:53.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:53.958+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f390c0763d0 con 0x7f39240d0e30 2023-10-19T10:37:54.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.133+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3924073990 con 0x7f39240d0e30 2023-10-19T10:37:54.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.134+0000 7f391b7fe640 1 -- 172.21.15.50:0/2598751575 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f390c035020 con 0x7f39240d0e30 2023-10-19T10:37:54.137 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:54.137 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:54.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f390406a050 msgr2=0x7f390406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f390406a050 0x7f390406c510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3914007fd0 tx=0x7f3914009040 comp rx=0 tx=0).stop 2023-10-19T10:37:54.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 msgr2=0x7f39240b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.139 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f39240b5cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f390c02b950 tx=0x7f390c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 shutdown_connections 2023-10-19T10:37:54.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f390406a050 0x7f390406c510 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f392415c570 0x7f39240c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39240d0e30 0x7f39240b5cb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 --2- 172.21.15.50:0/2598751575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39240cd920 0x7f39240cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.140 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 >> 172.21.15.50:0/2598751575 conn(0x7f3924089ff0 msgr2=0x7f39240ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:54.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 shutdown_connections 2023-10-19T10:37:54.141 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.136+0000 7f392937f640 1 -- 172.21.15.50:0/2598751575 wait complete. 2023-10-19T10:37:54.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:54.151 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:54.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.258+0000 7f5817727640 1 Processor -- start 2023-10-19T10:37:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.258+0000 7f5817727640 1 -- start start 2023-10-19T10:37:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5817727640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f581015c960 0x7f58100d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5817727640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581015d280 con 0x7f58100b4070 2023-10-19T10:37:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5817727640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581015d560 con 0x7f581015cdf0 2023-10-19T10:37:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5817727640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581015d840 con 0x7f581015c960 2023-10-19T10:37:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5814cc2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f581015c960 0x7f58100d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5814cc2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f581015c960 0x7f58100d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33176/0 (socket says 172.21.15.50:33176) 2023-10-19T10:37:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5814cc2640 1 -- 172.21.15.50:0/2108281152 learned_addr learned my addr 172.21.15.50:0/2108281152 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5814cc2640 1 -- 172.21.15.50:0/2108281152 >> v1:172.21.15.183:6789/0 conn(0x7f581015cdf0 legacy=0x7f58100cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5814cc2640 1 -- 172.21.15.50:0/2108281152 >> v1:172.21.15.50:6789/0 conn(0x7f58100b4070 legacy=0x7f58100b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.259+0000 7f5814cc2640 1 -- 172.21.15.50:0/2108281152 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581015db20 con 0x7f581015c960 2023-10-19T10:37:54.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.260+0000 7f5814cc2640 1 --2- 172.21.15.50:0/2108281152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f58100d2d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ae97bb841e93c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.260+0000 7f580ffff640 1 -- 172.21.15.50:0/2108281152 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580402f840 con 0x7f581015c960 2023-10-19T10:37:54.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.260+0000 7f580ffff640 1 -- 172.21.15.50:0/2108281152 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580402f9a0 con 0x7f581015c960 2023-10-19T10:37:54.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.260+0000 7f580ffff640 1 -- 172.21.15.50:0/2108281152 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580402fd30 con 0x7f581015c960 2023-10-19T10:37:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 -- 172.21.15.50:0/2108281152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 msgr2=0x7f58100d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 --2- 172.21.15.50:0/2108281152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f58100d2d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580402ba20 comp rx=0 tx=0).stop 2023-10-19T10:37:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 -- 172.21.15.50:0/2108281152 shutdown_connections 2023-10-19T10:37:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 --2- 172.21.15.50:0/2108281152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f58100d2d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 -- 172.21.15.50:0/2108281152 >> 172.21.15.50:0/2108281152 conn(0x7f581009b970 msgr2=0x7f58100d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 -- 172.21.15.50:0/2108281152 shutdown_connections 2023-10-19T10:37:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.261+0000 7f5817727640 1 -- 172.21.15.50:0/2108281152 wait complete. 2023-10-19T10:37:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.262+0000 7f5817727640 1 Processor -- start 2023-10-19T10:37:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.262+0000 7f5817727640 1 -- start start 2023-10-19T10:37:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5817727640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58100b4070 0x7f58100c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5817727640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f581015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5817727640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58100c7ad0 0x7f58100c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5817727640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100c8d80 con 0x7f581015c960 2023-10-19T10:37:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5817727640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100c84c0 con 0x7f58100c7ad0 2023-10-19T10:37:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5817727640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b6d50 con 0x7f58100b4070 2023-10-19T10:37:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f581015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f581015b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33182/0 (socket says 172.21.15.50:33182) 2023-10-19T10:37:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 -- 172.21.15.50:0/946790990 learned_addr learned my addr 172.21.15.50:0/946790990 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f58154c3640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58100b4070 0x7f58100c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5815cc4640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58100c7ad0 0x7f58100c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 -- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58100b4070 msgr2=0x7f58100c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58100b4070 0x7f58100c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 -- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58100c7ad0 msgr2=0x7f58100c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58100c7ad0 0x7f58100c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5814cc2640 1 -- 172.21.15.50:0/946790990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040094d0 con 0x7f581015c960 2023-10-19T10:37:54.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f5815cc4640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58100c7ad0 0x7f58100c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:54.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.263+0000 7f58154c3640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58100b4070 0x7f58100c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:54.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.264+0000 7f5814cc2640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f581015b0b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f580402ba60 tx=0x7f5804004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.264+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580402f810 con 0x7f581015c960 2023-10-19T10:37:54.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.264+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804042480 con 0x7f581015c960 2023-10-19T10:37:54.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.264+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58100b6a70 con 0x7f581015c960 2023-10-19T10:37:54.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.264+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804042850 con 0x7f581015c960 2023-10-19T10:37:54.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.264+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58100c89a0 con 0x7f581015c960 2023-10-19T10:37:54.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.265+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58100d09b0 con 0x7f581015c960 2023-10-19T10:37:54.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.265+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f580402d3d0 con 0x7f581015c960 2023-10-19T10:37:54.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.266+0000 7f580dffb640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f57fc06a050 0x7f57fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.266+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f580403c070 con 0x7f581015c960 2023-10-19T10:37:54.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.266+0000 7f58154c3640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f57fc06a050 0x7f57fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.266+0000 7f58154c3640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f57fc06a050 0x7f57fc06c510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5808000fd0 tx=0x7f5808009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.270+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5804076650 con 0x7f581015c960 2023-10-19T10:37:54.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.445+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58100c8ec0 con 0x7f581015c960 2023-10-19T10:37:54.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.445+0000 7f580dffb640 1 -- 172.21.15.50:0/946790990 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58100c8ec0 con 0x7f581015c960 2023-10-19T10:37:54.448 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f57fc06a050 msgr2=0x7f57fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f57fc06a050 0x7f57fc06c510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5808000fd0 tx=0x7f5808009040 comp rx=0 tx=0).stop 2023-10-19T10:37:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 msgr2=0x7f581015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f581015b0b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f580402ba60 tx=0x7f5804004730 comp rx=0 tx=0).stop 2023-10-19T10:37:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 shutdown_connections 2023-10-19T10:37:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f57fc06a050 0x7f57fc06c510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58100c7ad0 0x7f58100c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f581015c960 0x7f581015b0b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.447+0000 7f5817727640 1 --2- 172.21.15.50:0/946790990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58100b4070 0x7f58100c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.448+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 >> 172.21.15.50:0/946790990 conn(0x7f581009b970 msgr2=0x7f5810072390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.448+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 shutdown_connections 2023-10-19T10:37:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.448+0000 7f5817727640 1 -- 172.21.15.50:0/946790990 wait complete. 2023-10-19T10:37:54.461 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:54.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.568+0000 7efe19a28640 1 Processor -- start 2023-10-19T10:37:54.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.568+0000 7efe19a28640 1 -- start start 2023-10-19T10:37:54.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe19a28640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b38f0 0x7efe140d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe19a28640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efe140b3090 0x7efe14160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe19a28640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7efe140b3cd0 0x7efe14161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe19a28640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe140b0190 con 0x7efe140b3cd0 2023-10-19T10:37:54.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe19a28640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe140af250 con 0x7efe140b3090 2023-10-19T10:37:54.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe19a28640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140af510 con 0x7efe140b38f0 2023-10-19T10:37:54.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7efe140b3cd0 0x7efe14161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7efe140b3cd0 0x7efe14161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33198/0 (socket says 172.21.15.50:33198) 2023-10-19T10:37:54.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 -- 172.21.15.50:0/2225177677 learned_addr learned my addr 172.21.15.50:0/2225177677 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:54.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe12ffd640 1 --2- 172.21.15.50:0/2225177677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b38f0 0x7efe140d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe127fc640 1 --2- 172.21.15.50:0/2225177677 >> v2:172.21.15.183:3300/0 conn(0x7efe140b3090 0x7efe14160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 -- 172.21.15.50:0/2225177677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b38f0 msgr2=0x7efe140d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:54.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 --2- 172.21.15.50:0/2225177677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b38f0 0x7efe140d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 -- 172.21.15.50:0/2225177677 >> v2:172.21.15.183:3300/0 conn(0x7efe140b3090 msgr2=0x7efe14160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:54.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.569+0000 7efe137fe640 1 --2- 172.21.15.50:0/2225177677 >> v2:172.21.15.183:3300/0 conn(0x7efe140b3090 0x7efe14160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.570+0000 7efe137fe640 1 -- 172.21.15.50:0/2225177677 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140af7f0 con 0x7efe140b3cd0 2023-10-19T10:37:54.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.570+0000 7efe137fe640 1 --2- 172.21.15.50:0/2225177677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b3cd0 0x7efe14161040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe1408b140 tx=0x7efe0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85cd6c7ccc1381d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.570+0000 7efe11ffb640 1 -- 172.21.15.50:0/2225177677 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c031870 con 0x7efe140b3cd0 2023-10-19T10:37:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.570+0000 7efe11ffb640 1 -- 172.21.15.50:0/2225177677 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c0319d0 con 0x7efe140b3cd0 2023-10-19T10:37:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.570+0000 7efe11ffb640 1 -- 172.21.15.50:0/2225177677 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c031d60 con 0x7efe140b3cd0 2023-10-19T10:37:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 -- 172.21.15.50:0/2225177677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b3cd0 msgr2=0x7efe14161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 --2- 172.21.15.50:0/2225177677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b3cd0 0x7efe14161040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe1408b140 tx=0x7efe0c02b910 comp rx=0 tx=0).stop 2023-10-19T10:37:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 -- 172.21.15.50:0/2225177677 shutdown_connections 2023-10-19T10:37:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 --2- 172.21.15.50:0/2225177677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b3cd0 0x7efe14161040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 --2- 172.21.15.50:0/2225177677 >> v2:172.21.15.183:3300/0 conn(0x7efe140b3090 0x7efe14160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 --2- 172.21.15.50:0/2225177677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b38f0 0x7efe140d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 -- 172.21.15.50:0/2225177677 >> 172.21.15.50:0/2225177677 conn(0x7efe1409b960 msgr2=0x7efe140d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 -- 172.21.15.50:0/2225177677 shutdown_connections 2023-10-19T10:37:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.571+0000 7efe19a28640 1 -- 172.21.15.50:0/2225177677 wait complete. 2023-10-19T10:37:54.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.572+0000 7efe19a28640 1 Processor -- start 2023-10-19T10:37:54.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 -- start start 2023-10-19T10:37:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe140b3090 0x7efe14156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 0x7efe141e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b3cd0 0x7efe141e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe141576c0 con 0x7efe140b38f0 2023-10-19T10:37:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe141579a0 con 0x7efe140b3090 2023-10-19T10:37:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe19a28640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe14157c80 con 0x7efe140b3cd0 2023-10-19T10:37:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe127fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 0x7efe141e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe127fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 0x7efe141e55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33202/0 (socket says 172.21.15.50:33202) 2023-10-19T10:37:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.573+0000 7efe127fc640 1 -- 172.21.15.50:0/556460338 learned_addr learned my addr 172.21.15.50:0/556460338 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe12ffd640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe140b3090 0x7efe14156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe137fe640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b3cd0 0x7efe141e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe127fc640 1 -- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b3cd0 msgr2=0x7efe141e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe127fc640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b3cd0 0x7efe141e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe127fc640 1 -- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe140b3090 msgr2=0x7efe14156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe127fc640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe140b3090 0x7efe14156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe127fc640 1 -- 172.21.15.50:0/556460338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0094d0 con 0x7efe140b38f0 2023-10-19T10:37:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe127fc640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 0x7efe141e55e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00d7d0 tx=0x7efdfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0096a0 con 0x7efe140b38f0 2023-10-19T10:37:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc009800 con 0x7efe140b38f0 2023-10-19T10:37:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc009b90 con 0x7efe140b38f0 2023-10-19T10:37:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1415e250 con 0x7efe140b38f0 2023-10-19T10:37:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.574+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14157290 con 0x7efe140b38f0 2023-10-19T10:37:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.575+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1415e0b0 con 0x7efe140b38f0 2023-10-19T10:37:54.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.575+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efdfc009cf0 con 0x7efe140b38f0 2023-10-19T10:37:54.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.576+0000 7efe0bfff640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efdf406a050 0x7efdf406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.576+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efdfc088e30 con 0x7efe140b38f0 2023-10-19T10:37:54.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.576+0000 7efe12ffd640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efdf406a050 0x7efdf406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.577+0000 7efe12ffd640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efdf406a050 0x7efdf406c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe04009ab0 tx=0x7efe04003990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.580+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7efe1415e0b0 con 0x7efe140b38f0 2023-10-19T10:37:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.757+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efe1415e0b0 con 0x7efe140b38f0 2023-10-19T10:37:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.757+0000 7efe0bfff640 1 -- 172.21.15.50:0/556460338 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7efe1415e0b0 con 0x7efe140b38f0 2023-10-19T10:37:54.760 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efdf406a050 msgr2=0x7efdf406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efdf406a050 0x7efdf406c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe04009ab0 tx=0x7efe04003990 comp rx=0 tx=0).stop 2023-10-19T10:37:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 msgr2=0x7efe141e55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 0x7efe141e55e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00d7d0 tx=0x7efdfc00dc90 comp rx=0 tx=0).stop 2023-10-19T10:37:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 shutdown_connections 2023-10-19T10:37:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efdf406a050 0x7efdf406c510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe140b3cd0 0x7efe141e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe140b38f0 0x7efe141e55e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 --2- 172.21.15.50:0/556460338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe140b3090 0x7efe14156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.759+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 >> 172.21.15.50:0/556460338 conn(0x7efe1409b960 msgr2=0x7efe140cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.760+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 shutdown_connections 2023-10-19T10:37:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.760+0000 7efe19a28640 1 -- 172.21.15.50:0/556460338 wait complete. 2023-10-19T10:37:54.773 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:54.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.880+0000 7f7d05cc4640 1 Processor -- start 2023-10-19T10:37:54.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.880+0000 7f7d05cc4640 1 -- start start 2023-10-19T10:37:54.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.880+0000 7f7d05cc4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0015cdf0 0x7f7d001675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.880+0000 7f7d05cc4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d000d3480 con 0x7f7d000b4070 2023-10-19T10:37:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.880+0000 7f7d05cc4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0015d280 con 0x7f7d0015c960 2023-10-19T10:37:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.880+0000 7f7d05cc4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0015d560 con 0x7f7d0015cdf0 2023-10-19T10:37:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cff7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f7d000b4070 0x7f7d000b4450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44764/0 (socket says 172.21.15.50:44764) 2023-10-19T10:37:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cff7fe640 1 -- 172.21.15.50:0/2304496069 learned_addr learned my addr 172.21.15.50:0/2304496069 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfffff640 1 --2- 172.21.15.50:0/2304496069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0015cdf0 0x7f7d001675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499767001 0 0) 0x7f7d000d3480 con 0x7f7d000b4070 2023-10-19T10:37:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce8003520 con 0x7f7d000b4070 2023-10-19T10:37:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420122306 0 0) 0x7f7d0015d280 con 0x7f7d0015c960 2023-10-19T10:37:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d000d3480 con 0x7f7d0015c960 2023-10-19T10:37:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfffff640 1 -- 172.21.15.50:0/2304496069 >> v1:172.21.15.183:6789/0 conn(0x7f7d0015c960 legacy=0x7f7d000d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfffff640 1 -- 172.21.15.50:0/2304496069 >> v1:172.21.15.50:6789/0 conn(0x7f7d000b4070 legacy=0x7f7d000b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.881+0000 7f7cfffff640 1 -- 172.21.15.50:0/2304496069 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0015d840 con 0x7f7d0015cdf0 2023-10-19T10:37:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.882+0000 7f7cfffff640 1 --2- 172.21.15.50:0/2304496069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0015cdf0 0x7f7d001675b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009dc0 tx=0x7f7cf802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3c6bddc8afdd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.882+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf802ccd0 con 0x7f7d0015cdf0 2023-10-19T10:37:54.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.882+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf802ce30 con 0x7f7d0015cdf0 2023-10-19T10:37:54.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.882+0000 7f7cfe7fc640 1 -- 172.21.15.50:0/2304496069 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8034640 con 0x7f7d0015cdf0 2023-10-19T10:37:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 -- 172.21.15.50:0/2304496069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0015cdf0 msgr2=0x7f7d001675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/2304496069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0015cdf0 0x7f7d001675b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009dc0 tx=0x7f7cf802bf20 comp rx=0 tx=0).stop 2023-10-19T10:37:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 -- 172.21.15.50:0/2304496069 shutdown_connections 2023-10-19T10:37:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/2304496069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d0015cdf0 0x7f7d001675b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 -- 172.21.15.50:0/2304496069 >> 172.21.15.50:0/2304496069 conn(0x7f7d0009b970 msgr2=0x7f7d000d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 -- 172.21.15.50:0/2304496069 shutdown_connections 2023-10-19T10:37:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.883+0000 7f7d05cc4640 1 -- 172.21.15.50:0/2304496069 wait complete. 2023-10-19T10:37:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.884+0000 7f7d05cc4640 1 Processor -- start 2023-10-19T10:37:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.884+0000 7f7d05cc4640 1 -- start start 2023-10-19T10:37:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7d05cc4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d000b4070 0x7f7d000c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7d05cc4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0015c960 0x7f7d0015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7d05cc4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 0x7f7d00153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7d05cc4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0015e2a0 con 0x7f7d0015f570 2023-10-19T10:37:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7d05cc4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000c7250 con 0x7f7d0015c960 2023-10-19T10:37:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7d05cc4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0015e4d0 con 0x7f7d000b4070 2023-10-19T10:37:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 0x7f7d00153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 0x7f7d00153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33212/0 (socket says 172.21.15.50:33212) 2023-10-19T10:37:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 -- 172.21.15.50:0/815727454 learned_addr learned my addr 172.21.15.50:0/815727454 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cff7fe640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d000b4070 0x7f7d000c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfeffd640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0015c960 0x7f7d0015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 -- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d000b4070 msgr2=0x7f7d000c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d000b4070 0x7f7d000c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 -- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0015c960 msgr2=0x7f7d0015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0015c960 0x7f7d0015dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 -- 172.21.15.50:0/815727454 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8009a70 con 0x7f7d0015f570 2023-10-19T10:37:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.885+0000 7f7cfffff640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 0x7f7d00153cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7cf802dfd0 tx=0x7f7cf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.886+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8007000 con 0x7f7d0015f570 2023-10-19T10:37:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.886+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf803e630 con 0x7f7d0015f570 2023-10-19T10:37:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.886+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0015e7b0 con 0x7f7d0015f570 2023-10-19T10:37:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.886+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf803e9e0 con 0x7f7d0015f570 2023-10-19T10:37:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.886+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0015ed70 con 0x7f7d0015f570 2023-10-19T10:37:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.887+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0004e420 con 0x7f7d0015f570 2023-10-19T10:37:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.887+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7cf803b050 con 0x7f7d0015f570 2023-10-19T10:37:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.887+0000 7f7ce7fff640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7cec06a050 0x7f7cec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.888+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7cf80ab0d0 con 0x7f7d0015f570 2023-10-19T10:37:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.888+0000 7f7cff7fe640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7cec06a050 0x7f7cec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.888+0000 7f7cff7fe640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7cec06a050 0x7f7cec06c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0001000 tx=0x7f7cf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:54.892+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7cf8076650 con 0x7f7d0015f570 2023-10-19T10:37:55.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.066+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d0008c720 con 0x7f7d0015f570 2023-10-19T10:37:55.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.066+0000 7f7ce7fff640 1 -- 172.21.15.50:0/815727454 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7cf8039020 con 0x7f7d0015f570 2023-10-19T10:37:55.069 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:55.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7cec06a050 msgr2=0x7f7cec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7cec06a050 0x7f7cec06c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0001000 tx=0x7f7cf0009040 comp rx=0 tx=0).stop 2023-10-19T10:37:55.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 msgr2=0x7f7d00153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 0x7f7d00153cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7cf802dfd0 tx=0x7f7cf8004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:55.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 shutdown_connections 2023-10-19T10:37:55.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7cec06a050 0x7f7cec06c510 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d0015f570 0x7f7d00153cb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d0015c960 0x7f7d0015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 --2- 172.21.15.50:0/815727454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d000b4070 0x7f7d000c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 >> 172.21.15.50:0/815727454 conn(0x7f7d0009b970 msgr2=0x7f7d0006a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:55.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 shutdown_connections 2023-10-19T10:37:55.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.068+0000 7f7d05cc4640 1 -- 172.21.15.50:0/815727454 wait complete. 2023-10-19T10:37:55.081 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.188+0000 7faec5949640 1 Processor -- start 2023-10-19T10:37:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.188+0000 7faec5949640 1 -- start start 2023-10-19T10:37:55.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faec5949640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faec00b38f0 0x7faec00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faec5949640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec00af250 con 0x7faec0161210 2023-10-19T10:37:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faec5949640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec00af510 con 0x7faec00b3090 2023-10-19T10:37:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faec5949640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec00b0190 con 0x7faec00b38f0 2023-10-19T10:37:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faebf7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7faec0161210 0x7faec00b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44776/0 (socket says 172.21.15.50:44776) 2023-10-19T10:37:55.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faebf7fe640 1 -- 172.21.15.50:0/339040784 learned_addr learned my addr 172.21.15.50:0/339040784 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:55.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355230174 0 0) 0x7faec00af250 con 0x7faec0161210 2023-10-19T10:37:55.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faebeffd640 1 --2- 172.21.15.50:0/339040784 >> v2:172.21.15.183:3300/0 conn(0x7faec00b38f0 0x7faec00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.189+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea8003520 con 0x7faec0161210 2023-10-19T10:37:55.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899218433 0 0) 0x7faea8003520 con 0x7faec0161210 2023-10-19T10:37:55.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 >> v2:172.21.15.183:3300/0 conn(0x7faec00b38f0 msgr2=0x7faec00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 --2- 172.21.15.50:0/339040784 >> v2:172.21.15.183:3300/0 conn(0x7faec00b38f0 0x7faec00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 >> v1:172.21.15.183:6789/0 conn(0x7faec00b3090 legacy=0x7faec0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec00af7f0 con 0x7faec0161210 2023-10-19T10:37:55.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7faeb8001670 con 0x7faec0161210 2023-10-19T10:37:55.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 >> v1:172.21.15.50:6789/0 conn(0x7faec0161210 legacy=0x7faec00b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea8003970 0x7faea8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faea8004370 0x7faea8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8005040 0x7faea800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b0190 con 0x7faea8004370 2023-10-19T10:37:55.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea800bfa0 con 0x7faea8003970 2023-10-19T10:37:55.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea800c160 con 0x7faea8005040 2023-10-19T10:37:55.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebe7fc640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faea8004370 0x7faea8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebeffd640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea8003970 0x7faea8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebf7fe640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8005040 0x7faea800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebe7fc640 1 -- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8005040 msgr2=0x7faea800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebe7fc640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8005040 0x7faea800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebe7fc640 1 -- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea8003970 msgr2=0x7faea8003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebe7fc640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faea8003970 0x7faea8003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebe7fc640 1 -- 172.21.15.50:0/339040784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8003520 con 0x7faea8004370 2023-10-19T10:37:55.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebf7fe640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8005040 0x7faea800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:55.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.190+0000 7faebf7fe640 1 -- 172.21.15.50:0/339040784 reap_dead start 2023-10-19T10:37:55.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.191+0000 7faebe7fc640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faea8004370 0x7faea8004840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faeb4001860 tx=0x7faea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df6a0621d5e8d6d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.191+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb402f050 con 0x7faea8004370 2023-10-19T10:37:55.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.191+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb4033610 con 0x7faea8004370 2023-10-19T10:37:55.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.191+0000 7faebdffb640 1 -- 172.21.15.50:0/339040784 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb40339c0 con 0x7faea8004370 2023-10-19T10:37:55.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 -- 172.21.15.50:0/339040784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faea8004370 msgr2=0x7faea8004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faea8004370 0x7faea8004840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faeb4001860 tx=0x7faea80057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 -- 172.21.15.50:0/339040784 shutdown_connections 2023-10-19T10:37:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 --2- 172.21.15.50:0/339040784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faea8004370 0x7faea8004840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 -- 172.21.15.50:0/339040784 >> 172.21.15.50:0/339040784 conn(0x7faec009b960 msgr2=0x7faec00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 -- 172.21.15.50:0/339040784 shutdown_connections 2023-10-19T10:37:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.192+0000 7faec5949640 1 -- 172.21.15.50:0/339040784 wait complete. 2023-10-19T10:37:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.193+0000 7faec5949640 1 Processor -- start 2023-10-19T10:37:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.193+0000 7faec5949640 1 -- start start 2023-10-19T10:37:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faec5949640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8004370 0x7faec00b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faec5949640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 0x7faec00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faec5949640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec015cf00 0x7faec015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faec5949640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec015d8c0 con 0x7faec0158250 2023-10-19T10:37:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faec5949640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00cefb0 con 0x7faec015cf00 2023-10-19T10:37:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faec5949640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec00cf290 con 0x7faea8004370 2023-10-19T10:37:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 0x7faec00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 0x7faec00ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33228/0 (socket says 172.21.15.50:33228) 2023-10-19T10:37:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 -- 172.21.15.50:0/2787315091 learned_addr learned my addr 172.21.15.50:0/2787315091 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebf7fe640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec015cf00 0x7faec015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebeffd640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8004370 0x7faec00b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 -- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8004370 msgr2=0x7faec00b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8004370 0x7faec00b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 -- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec015cf00 msgr2=0x7faec015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec015cf00 0x7faec015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 -- 172.21.15.50:0/2787315091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec00cf570 con 0x7faec0158250 2023-10-19T10:37:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebf7fe640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec015cf00 0x7faec015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.194+0000 7faebe7fc640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 0x7faec00ce760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faeb402bd30 tx=0x7faeb4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.195+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb403d910 con 0x7faec0158250 2023-10-19T10:37:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.195+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb403da70 con 0x7faec0158250 2023-10-19T10:37:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.195+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb403de20 con 0x7faec0158250 2023-10-19T10:37:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.195+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec00cf850 con 0x7faec0158250 2023-10-19T10:37:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.195+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec01586d0 con 0x7faec0158250 2023-10-19T10:37:55.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.196+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec004e420 con 0x7faec0158250 2023-10-19T10:37:55.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.196+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faeb403c3a0 con 0x7faec0158250 2023-10-19T10:37:55.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.197+0000 7faea7fff640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faeac06a080 0x7faeac06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.197+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faeb40aa630 con 0x7faec0158250 2023-10-19T10:37:55.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.197+0000 7faebeffd640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faeac06a080 0x7faeac06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.197+0000 7faebeffd640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faeac06a080 0x7faeac06c540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faec00b57e0 tx=0x7faeb000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.201+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7faeb4079510 con 0x7faec0158250 2023-10-19T10:37:55.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.379+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faec015d8c0 con 0x7faec0158250 2023-10-19T10:37:55.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.379+0000 7faea7fff640 1 -- 172.21.15.50:0/2787315091 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7faec015d8c0 con 0x7faec0158250 2023-10-19T10:37:55.382 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:55.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faeac06a080 msgr2=0x7faeac06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faeac06a080 0x7faeac06c540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faec00b57e0 tx=0x7faeb000b040 comp rx=0 tx=0).stop 2023-10-19T10:37:55.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 msgr2=0x7faec00ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 0x7faec00ce760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faeb402bd30 tx=0x7faeb4001230 comp rx=0 tx=0).stop 2023-10-19T10:37:55.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 shutdown_connections 2023-10-19T10:37:55.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faeac06a080 0x7faeac06c540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec015cf00 0x7faec015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faec0158250 0x7faec00ce760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 --2- 172.21.15.50:0/2787315091 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faea8004370 0x7faec00b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 >> 172.21.15.50:0/2787315091 conn(0x7faec009b960 msgr2=0x7faec00b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:55.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 shutdown_connections 2023-10-19T10:37:55.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.381+0000 7faec5949640 1 -- 172.21.15.50:0/2787315091 wait complete. 2023-10-19T10:37:55.395 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:55.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.502+0000 7f98941b2640 1 Processor -- start 2023-10-19T10:37:55.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.502+0000 7f98941b2640 1 -- start start 2023-10-19T10:37:55.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f98941b2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f988c0b26e0 0x7f988c06f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f98941b2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f988c145880 0x7f988c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f98941b2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c0701b0 con 0x7f988c0ce560 2023-10-19T10:37:55.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f98941b2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c0ce9d0 con 0x7f988c145880 2023-10-19T10:37:55.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f98941b2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c06f910 con 0x7f988c0b26e0 2023-10-19T10:37:55.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f988c145880 0x7f988c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f988c145880 0x7f988c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33236/0 (socket says 172.21.15.50:33236) 2023-10-19T10:37:55.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989274f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f988c0ce560 0x7f988c1600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44778/0 (socket says 172.21.15.50:44778) 2023-10-19T10:37:55.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 -- 172.21.15.50:0/2394209115 learned_addr learned my addr 172.21.15.50:0/2394209115 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:55.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f9891f4e640 1 --2- 172.21.15.50:0/2394209115 >> v2:172.21.15.183:3300/0 conn(0x7f988c0b26e0 0x7f988c06f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 -- 172.21.15.50:0/2394209115 >> v2:172.21.15.183:3300/0 conn(0x7f988c0b26e0 msgr2=0x7f988c06f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 --2- 172.21.15.50:0/2394209115 >> v2:172.21.15.183:3300/0 conn(0x7f988c0b26e0 0x7f988c06f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 -- 172.21.15.50:0/2394209115 >> v1:172.21.15.50:6789/0 conn(0x7f988c0ce560 legacy=0x7f988c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.503+0000 7f989174d640 1 -- 172.21.15.50:0/2394209115 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c06fbf0 con 0x7f988c145880 2023-10-19T10:37:55.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.504+0000 7f989174d640 1 --2- 172.21.15.50:0/2394209115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c145880 0x7f988c0ce020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f988c0cdfd0 tx=0x7f988002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ad01b3abe3ad0c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.504+0000 7f9890f4c640 1 -- 172.21.15.50:0/2394209115 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880031870 con 0x7f988c145880 2023-10-19T10:37:55.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.504+0000 7f9890f4c640 1 -- 172.21.15.50:0/2394209115 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98800319d0 con 0x7f988c145880 2023-10-19T10:37:55.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.504+0000 7f9890f4c640 1 -- 172.21.15.50:0/2394209115 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880031d60 con 0x7f988c145880 2023-10-19T10:37:55.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 -- 172.21.15.50:0/2394209115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c145880 msgr2=0x7f988c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 --2- 172.21.15.50:0/2394209115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c145880 0x7f988c0ce020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f988c0cdfd0 tx=0x7f988002b910 comp rx=0 tx=0).stop 2023-10-19T10:37:55.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 -- 172.21.15.50:0/2394209115 shutdown_connections 2023-10-19T10:37:55.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 --2- 172.21.15.50:0/2394209115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c145880 0x7f988c0ce020 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 --2- 172.21.15.50:0/2394209115 >> v2:172.21.15.183:3300/0 conn(0x7f988c0b26e0 0x7f988c06f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 -- 172.21.15.50:0/2394209115 >> 172.21.15.50:0/2394209115 conn(0x7f988c0ccf60 msgr2=0x7f988c0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:55.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 -- 172.21.15.50:0/2394209115 shutdown_connections 2023-10-19T10:37:55.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.505+0000 7f98941b2640 1 -- 172.21.15.50:0/2394209115 wait complete. 2023-10-19T10:37:55.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.506+0000 7f98941b2640 1 Processor -- start 2023-10-19T10:37:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.506+0000 7f98941b2640 1 -- start start 2023-10-19T10:37:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f98941b2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 0x7f988c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f98941b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f988c0ce560 0x7f988c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f98941b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f988c145880 0x7f988c0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f98941b2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0b41d0 con 0x7f988c0b26e0 2023-10-19T10:37:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f98941b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0b5040 con 0x7f988c0ce560 2023-10-19T10:37:55.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f98941b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0b4710 con 0x7f988c145880 2023-10-19T10:37:55.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 0x7f988c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 0x7f988c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33252/0 (socket says 172.21.15.50:33252) 2023-10-19T10:37:55.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 -- 172.21.15.50:0/495316499 learned_addr learned my addr 172.21.15.50:0/495316499 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:55.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f989274f640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f988c145880 0x7f988c0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f989174d640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f988c0ce560 0x7f988c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 -- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f988c145880 msgr2=0x7f988c0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f988c145880 0x7f988c0c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 -- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f988c0ce560 msgr2=0x7f988c0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f988c0ce560 0x7f988c0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.507+0000 7f9891f4e640 1 -- 172.21.15.50:0/495316499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800094d0 con 0x7f988c0b26e0 2023-10-19T10:37:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.508+0000 7f9891f4e640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 0x7f988c0c8aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f988400e870 tx=0x7f988400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.508+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840094f0 con 0x7f988c0b26e0 2023-10-19T10:37:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.508+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988400c940 con 0x7f988c0b26e0 2023-10-19T10:37:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.508+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c0b49f0 con 0x7f988c0b26e0 2023-10-19T10:37:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.508+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400ccf0 con 0x7f988c0b26e0 2023-10-19T10:37:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.508+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c0ca890 con 0x7f988c0b26e0 2023-10-19T10:37:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.509+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c04e420 con 0x7f988c0b26e0 2023-10-19T10:37:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.509+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9884013470 con 0x7f988c0b26e0 2023-10-19T10:37:55.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.510+0000 7f988a7fc640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f987806a080 0x7f987806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.510+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9884088970 con 0x7f988c0b26e0 2023-10-19T10:37:55.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.510+0000 7f989174d640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f987806a080 0x7f987806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.510+0000 7f989174d640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f987806a080 0x7f987806c540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9880036000 tx=0x7f988003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.521 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.514+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9884053d30 con 0x7f988c0b26e0 2023-10-19T10:37:55.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.687+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f988c0b6460 con 0x7f988c0b26e0 2023-10-19T10:37:55.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.688+0000 7f988a7fc640 1 -- 172.21.15.50:0/495316499 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9884057350 con 0x7f988c0b26e0 2023-10-19T10:37:55.691 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:55.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f987806a080 msgr2=0x7f987806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f987806a080 0x7f987806c540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9880036000 tx=0x7f988003c040 comp rx=0 tx=0).stop 2023-10-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 msgr2=0x7f988c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 0x7f988c0c8aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f988400e870 tx=0x7f988400ed30 comp rx=0 tx=0).stop 2023-10-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 shutdown_connections 2023-10-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f987806a080 0x7f987806c540 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f988c145880 0x7f988c0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f988c0ce560 0x7f988c0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 --2- 172.21.15.50:0/495316499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988c0b26e0 0x7f988c0c8aa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.689+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 >> 172.21.15.50:0/495316499 conn(0x7f988c0ccf60 msgr2=0x7f988c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:55.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.690+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 shutdown_connections 2023-10-19T10:37:55.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.690+0000 7f98941b2640 1 -- 172.21.15.50:0/495316499 wait complete. 2023-10-19T10:37:55.703 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:37:55.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.808+0000 7f13d8ba0640 1 Processor -- start 2023-10-19T10:37:55.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.808+0000 7f13d8ba0640 1 -- start start 2023-10-19T10:37:55.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d8ba0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 0x7f13d415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d8ba0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40cf670 con 0x7f13d40b7ab0 2023-10-19T10:37:55.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d8ba0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d4160e50 con 0x7f13d415c3c0 2023-10-19T10:37:55.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d8ba0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d415c7a0 con 0x7f13d40cd500 2023-10-19T10:37:55.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d3fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f13d40b7ab0 0x7f13d40d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44792/0 (socket says 172.21.15.50:44792) 2023-10-19T10:37:55.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d3fff640 1 -- 172.21.15.50:0/640473543 learned_addr learned my addr 172.21.15.50:0/640473543 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:55.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d2ffd640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 0x7f13d415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.809+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019977788 0 0) 0x7f13d40cf670 con 0x7f13d40b7ab0 2023-10-19T10:37:55.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc003520 con 0x7f13d40b7ab0 2023-10-19T10:37:55.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224467852 0 0) 0x7f13bc003520 con 0x7f13d40b7ab0 2023-10-19T10:37:55.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 msgr2=0x7f13d415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 0x7f13d415feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 >> v1:172.21.15.183:6789/0 conn(0x7f13d415c3c0 legacy=0x7f13d40ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d41605b0 con 0x7f13d40b7ab0 2023-10-19T10:37:55.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f13cc0019f0 con 0x7f13d40b7ab0 2023-10-19T10:37:55.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 >> v1:172.21.15.50:6789/0 conn(0x7f13d40b7ab0 legacy=0x7f13d40d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13bc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 0x7f13bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc004d00 0x7f13bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d415c7a0 con 0x7f13bc0039a0 2023-10-19T10:37:55.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc00ba60 con 0x7f13bc004d00 2023-10-19T10:37:55.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc00bc20 con 0x7f13d40cd500 2023-10-19T10:37:55.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d37fe640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13bc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d2ffd640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 0x7f13bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d3fff640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc004d00 0x7f13bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d37fe640 1 -- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 msgr2=0x7f13bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d37fe640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40cd500 0x7f13bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d37fe640 1 -- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc004d00 msgr2=0x7f13bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:55.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d37fe640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc004d00 0x7f13bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d37fe640 1 -- 172.21.15.50:0/640473543 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc003520 con 0x7f13bc0039a0 2023-10-19T10:37:55.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.810+0000 7f13d3fff640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc004d00 0x7f13bc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:55.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.811+0000 7f13d37fe640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13bc003d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f13c4001910 tx=0x7f13bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39cce6f88fa7657 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.811+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4032650 con 0x7f13bc0039a0 2023-10-19T10:37:55.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.811+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c40327b0 con 0x7f13bc0039a0 2023-10-19T10:37:55.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.811+0000 7f13d27fc640 1 -- 172.21.15.50:0/640473543 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4032b40 con 0x7f13bc0039a0 2023-10-19T10:37:55.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d8ba0640 1 -- 172.21.15.50:0/640473543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 msgr2=0x7f13bc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/640473543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13bc003d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f13c4001910 tx=0x7f13bc0057f0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d3fff640 1 -- 172.21.15.50:0/640473543 reap_dead start 2023-10-19T10:37:55.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d8ba0640 1 -- 172.21.15.50:0/640473543 shutdown_connections 2023-10-19T10:37:55.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d8ba0640 1 -- 172.21.15.50:0/640473543 >> 172.21.15.50:0/640473543 conn(0x7f13d406a190 msgr2=0x7f13d406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:55.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d8ba0640 1 -- 172.21.15.50:0/640473543 shutdown_connections 2023-10-19T10:37:55.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.812+0000 7f13d8ba0640 1 -- 172.21.15.50:0/640473543 wait complete. 2023-10-19T10:37:55.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.813+0000 7f13d8ba0640 1 Processor -- start 2023-10-19T10:37:55.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.813+0000 7f13d8ba0640 1 -- start start 2023-10-19T10:37:55.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d8ba0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13d4158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d8ba0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13d40c91f0 0x7f13d415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d8ba0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40b6930 0x7f13d40b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d8ba0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40b72f0 con 0x7f13bc0039a0 2023-10-19T10:37:55.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d8ba0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41f1960 con 0x7f13d40c91f0 2023-10-19T10:37:55.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d8ba0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c9810 con 0x7f13d40b6930 2023-10-19T10:37:55.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13d4158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13d4158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33276/0 (socket says 172.21.15.50:33276) 2023-10-19T10:37:55.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 -- 172.21.15.50:0/81352822 learned_addr learned my addr 172.21.15.50:0/81352822 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:55.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d2ffd640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13d40c91f0 0x7f13d415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d3fff640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40b6930 0x7f13d40b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 -- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40b6930 msgr2=0x7f13d40b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40b6930 0x7f13d40b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 -- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13d40c91f0 msgr2=0x7f13d415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:55.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13d40c91f0 0x7f13d415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:55.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.814+0000 7f13d37fe640 1 -- 172.21.15.50:0/81352822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40c9af0 con 0x7f13bc0039a0 2023-10-19T10:37:55.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.815+0000 7f13d37fe640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13d4158be0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f13c4034040 tx=0x7f13c40012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.815+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4006850 con 0x7f13bc0039a0 2023-10-19T10:37:55.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.815+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c40069b0 con 0x7f13bc0039a0 2023-10-19T10:37:55.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.815+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d40c9dd0 con 0x7f13bc0039a0 2023-10-19T10:37:55.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.815+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4006d20 con 0x7f13bc0039a0 2023-10-19T10:37:55.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.815+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d415f710 con 0x7f13bc0039a0 2023-10-19T10:37:55.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.816+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d41f1c10 con 0x7f13bc0039a0 2023-10-19T10:37:55.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.816+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f13c4042070 con 0x7f13bc0039a0 2023-10-19T10:37:55.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.817+0000 7f13bbfff640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f13c006a050 0x7f13c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:55.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.817+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f13c40aaa30 con 0x7f13bc0039a0 2023-10-19T10:37:55.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.817+0000 7f13d2ffd640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f13c006a050 0x7f13c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:55.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.817+0000 7f13d2ffd640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f13c006a050 0x7f13c006c510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f13d40c8440 tx=0x7f13c8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:55.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:55.821+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f13c4075de0 con 0x7f13bc0039a0 2023-10-19T10:37:55.839 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:37:55.841 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:37:56.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.009+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f13d40c95d0 con 0x7f13bc0039a0 2023-10-19T10:37:56.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.010+0000 7f13bbfff640 1 -- 172.21.15.50:0/81352822 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f13d40c95d0 con 0x7f13bc0039a0 2023-10-19T10:37:56.013 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:37:56.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f13c006a050 msgr2=0x7f13c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f13c006a050 0x7f13c006c510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f13d40c8440 tx=0x7f13c8009190 comp rx=0 tx=0).stop 2023-10-19T10:37:56.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 msgr2=0x7f13d4158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13d4158be0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f13c4034040 tx=0x7f13c40012b0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 shutdown_connections 2023-10-19T10:37:56.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f13c006a050 0x7f13c006c510 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f13d40b6930 0x7f13d40b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13d40c91f0 0x7f13d415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 --2- 172.21.15.50:0/81352822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc0039a0 0x7f13d4158be0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 >> 172.21.15.50:0/81352822 conn(0x7f13d406a190 msgr2=0x7f13d40b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:56.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 shutdown_connections 2023-10-19T10:37:56.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.012+0000 7f13d8ba0640 1 -- 172.21.15.50:0/81352822 wait complete. 2023-10-19T10:37:56.025 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:37:56.026 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:56.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:56.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.185+0000 7fbe5e711640 1 Processor -- start 2023-10-19T10:37:56.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.185+0000 7fbe5e711640 1 -- start start 2023-10-19T10:37:56.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe5e711640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbe5815c570 0x7fbe580d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe5e711640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c950 0x7fbe580cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe5e711640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580cde60 con 0x7fbe5806b760 2023-10-19T10:37:56.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe5e711640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5806aa30 con 0x7fbe5815c570 2023-10-19T10:37:56.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe5e711640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5806b2d0 con 0x7fbe5815c950 2023-10-19T10:37:56.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbe5815c570 0x7fbe580d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbe5815c570 0x7fbe580d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33278/0 (socket says 172.21.15.50:33278) 2023-10-19T10:37:56.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 -- 172.21.15.50:0/933773820 learned_addr learned my addr 172.21.15.50:0/933773820 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:56.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 -- 172.21.15.50:0/933773820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c950 msgr2=0x7fbe580cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:56.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe577fe640 1 --2- 172.21.15.50:0/933773820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c950 0x7fbe580cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 --2- 172.21.15.50:0/933773820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c950 0x7fbe580cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 -- 172.21.15.50:0/933773820 >> v1:172.21.15.50:6789/0 conn(0x7fbe5806b760 legacy=0x7fbe580d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.186+0000 7fbe57fff640 1 -- 172.21.15.50:0/933773820 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580ce090 con 0x7fbe5815c570 2023-10-19T10:37:56.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.187+0000 7fbe57fff640 1 --2- 172.21.15.50:0/933773820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5815c570 0x7fbe580d0e30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009850 tx=0x7fbe4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785532796176874c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.187+0000 7fbe56ffd640 1 -- 172.21.15.50:0/933773820 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c004230 con 0x7fbe5815c570 2023-10-19T10:37:56.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.187+0000 7fbe56ffd640 1 -- 172.21.15.50:0/933773820 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c004390 con 0x7fbe5815c570 2023-10-19T10:37:56.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.187+0000 7fbe56ffd640 1 -- 172.21.15.50:0/933773820 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c031830 con 0x7fbe5815c570 2023-10-19T10:37:56.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 -- 172.21.15.50:0/933773820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5815c570 msgr2=0x7fbe580d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 --2- 172.21.15.50:0/933773820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5815c570 0x7fbe580d0e30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009850 tx=0x7fbe4c02b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 -- 172.21.15.50:0/933773820 shutdown_connections 2023-10-19T10:37:56.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 --2- 172.21.15.50:0/933773820 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c950 0x7fbe580cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 --2- 172.21.15.50:0/933773820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5815c570 0x7fbe580d0e30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 -- 172.21.15.50:0/933773820 >> 172.21.15.50:0/933773820 conn(0x7fbe58089ff0 msgr2=0x7fbe5809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:56.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 -- 172.21.15.50:0/933773820 shutdown_connections 2023-10-19T10:37:56.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.188+0000 7fbe5e711640 1 -- 172.21.15.50:0/933773820 wait complete. 2023-10-19T10:37:56.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.189+0000 7fbe5e711640 1 Processor -- start 2023-10-19T10:37:56.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.189+0000 7fbe5e711640 1 -- start start 2023-10-19T10:37:56.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5e711640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 0x7fbe5815a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5e711640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c570 0x7fbe5815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5e711640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe5815c950 0x7fbe5815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5e711640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c7f30 con 0x7fbe5806b760 2023-10-19T10:37:56.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5e711640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c7600 con 0x7fbe5815c950 2023-10-19T10:37:56.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5e711640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c78e0 con 0x7fbe5815c570 2023-10-19T10:37:56.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 0x7fbe5815a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 0x7fbe5815a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33282/0 (socket says 172.21.15.50:33282) 2023-10-19T10:37:56.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 -- 172.21.15.50:0/1405105069 learned_addr learned my addr 172.21.15.50:0/1405105069 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:56.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe577fe640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c570 0x7fbe5815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5ccae640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe5815c950 0x7fbe5815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 -- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c570 msgr2=0x7fbe5815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c570 0x7fbe5815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 -- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe5815c950 msgr2=0x7fbe5815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe5815c950 0x7fbe5815f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe57fff640 1 -- 172.21.15.50:0/1405105069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c009460 con 0x7fbe5806b760 2023-10-19T10:37:56.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe577fe640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c570 0x7fbe5815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:56.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.190+0000 7fbe5ccae640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe5815c950 0x7fbe5815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:56.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.191+0000 7fbe57fff640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 0x7fbe5815a410 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c02b910 tx=0x7fbe4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.191+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c031830 con 0x7fbe5806b760 2023-10-19T10:37:56.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.191+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c03f070 con 0x7fbe5806b760 2023-10-19T10:37:56.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.191+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe580c7bc0 con 0x7fbe5806b760 2023-10-19T10:37:56.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.191+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c031bd0 con 0x7fbe5806b760 2023-10-19T10:37:56.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.191+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580c83c0 con 0x7fbe5806b760 2023-10-19T10:37:56.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.192+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe580c85e0 con 0x7fbe5806b760 2023-10-19T10:37:56.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.192+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbe4c041030 con 0x7fbe5806b760 2023-10-19T10:37:56.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.193+0000 7fbe54ff9640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe4406a050 0x7fbe4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.193+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbe4c0aabe0 con 0x7fbe5806b760 2023-10-19T10:37:56.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.193+0000 7fbe577fe640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe4406a050 0x7fbe4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.193+0000 7fbe577fe640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe4406a050 0x7fbe4406c510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbe580c9520 tx=0x7fbe48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.197+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbe4c075f00 con 0x7fbe5806b760 2023-10-19T10:37:56.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.375+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe58073990 con 0x7fbe5806b760 2023-10-19T10:37:56.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.376+0000 7fbe54ff9640 1 -- 172.21.15.50:0/1405105069 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fbe4c079520 con 0x7fbe5806b760 2023-10-19T10:37:56.379 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:56.379 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:56.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe4406a050 msgr2=0x7fbe4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe4406a050 0x7fbe4406c510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbe580c9520 tx=0x7fbe48008040 comp rx=0 tx=0).stop 2023-10-19T10:37:56.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 msgr2=0x7fbe5815a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 0x7fbe5815a410 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c02b910 tx=0x7fbe4c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 shutdown_connections 2023-10-19T10:37:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe4406a050 0x7fbe4406c510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe5815c950 0x7fbe5815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe5815c570 0x7fbe5815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 --2- 172.21.15.50:0/1405105069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe5806b760 0x7fbe5815a410 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 >> 172.21.15.50:0/1405105069 conn(0x7fbe58089ff0 msgr2=0x7fbe5815ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:56.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 shutdown_connections 2023-10-19T10:37:56.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.378+0000 7fbe5e711640 1 -- 172.21.15.50:0/1405105069 wait complete. 2023-10-19T10:37:56.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:56.392 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.499+0000 7f7d57949640 1 Processor -- start 2023-10-19T10:37:56.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.500+0000 7f7d57949640 1 -- start start 2023-10-19T10:37:56.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.500+0000 7f7d57949640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7d500b4070 0x7f7d500b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.500+0000 7f7d57949640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015c960 0x7f7d500d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.500+0000 7f7d57949640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d500b4990 con 0x7f7d5015cdf0 2023-10-19T10:37:56.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.500+0000 7f7d57949640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5015d280 con 0x7f7d500b4070 2023-10-19T10:37:56.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.500+0000 7f7d57949640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5015d560 con 0x7f7d5015c960 2023-10-19T10:37:56.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7d500b4070 0x7f7d500b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7d500b4070 0x7f7d500b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33294/0 (socket says 172.21.15.50:33294) 2023-10-19T10:37:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 -- 172.21.15.50:0/1274328708 learned_addr learned my addr 172.21.15.50:0/1274328708 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 -- 172.21.15.50:0/1274328708 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015c960 msgr2=0x7f7d500d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 --2- 172.21.15.50:0/1274328708 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015c960 0x7f7d500d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 -- 172.21.15.50:0/1274328708 >> v1:172.21.15.50:6789/0 conn(0x7f7d5015cdf0 legacy=0x7f7d50160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.501+0000 7f7d556e5640 1 -- 172.21.15.50:0/1274328708 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5015d840 con 0x7f7d500b4070 2023-10-19T10:37:56.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d556e5640 1 --2- 172.21.15.50:0/1274328708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d500b4450 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009850 tx=0x7f7d4c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795df29ba84fb9d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d47fff640 1 -- 172.21.15.50:0/1274328708 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004230 con 0x7f7d500b4070 2023-10-19T10:37:56.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d47fff640 1 -- 172.21.15.50:0/1274328708 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c004390 con 0x7f7d500b4070 2023-10-19T10:37:56.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d47fff640 1 -- 172.21.15.50:0/1274328708 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c003810 con 0x7f7d500b4070 2023-10-19T10:37:56.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 -- 172.21.15.50:0/1274328708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 msgr2=0x7f7d500b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 --2- 172.21.15.50:0/1274328708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d500b4450 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009850 tx=0x7f7d4c02b960 comp rx=0 tx=0).stop 2023-10-19T10:37:56.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 -- 172.21.15.50:0/1274328708 shutdown_connections 2023-10-19T10:37:56.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 --2- 172.21.15.50:0/1274328708 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015c960 0x7f7d500d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 --2- 172.21.15.50:0/1274328708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d500b4450 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 -- 172.21.15.50:0/1274328708 >> 172.21.15.50:0/1274328708 conn(0x7f7d5009b970 msgr2=0x7f7d500d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:56.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 -- 172.21.15.50:0/1274328708 shutdown_connections 2023-10-19T10:37:56.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.502+0000 7f7d57949640 1 -- 172.21.15.50:0/1274328708 wait complete. 2023-10-19T10:37:56.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 Processor -- start 2023-10-19T10:37:56.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 -- start start 2023-10-19T10:37:56.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d50155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5015c960 0x7f7d5015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015e5e0 0x7f7d5015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5015efd0 con 0x7f7d500b4070 2023-10-19T10:37:56.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d500c6880 con 0x7f7d5015c960 2023-10-19T10:37:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d57949640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5015bc80 con 0x7f7d5015e5e0 2023-10-19T10:37:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d556e5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d50155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d556e5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d50155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33300/0 (socket says 172.21.15.50:33300) 2023-10-19T10:37:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.504+0000 7f7d556e5640 1 -- 172.21.15.50:0/3653907266 learned_addr learned my addr 172.21.15.50:0/3653907266 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d55ee6640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015e5e0 0x7f7d5015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d54ee4640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5015c960 0x7f7d5015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d556e5640 1 -- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015e5e0 msgr2=0x7f7d5015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d556e5640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015e5e0 0x7f7d5015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d556e5640 1 -- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5015c960 msgr2=0x7f7d5015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d556e5640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5015c960 0x7f7d5015aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d556e5640 1 -- 172.21.15.50:0/3653907266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c009460 con 0x7f7d500b4070 2023-10-19T10:37:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d54ee4640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5015c960 0x7f7d5015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d55ee6640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015e5e0 0x7f7d5015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d556e5640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d50155e60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009970 tx=0x7f7d4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c03e070 con 0x7f7d500b4070 2023-10-19T10:37:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c003bd0 con 0x7f7d500b4070 2023-10-19T10:37:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5015b3e0 con 0x7f7d500b4070 2023-10-19T10:37:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c006040 con 0x7f7d500b4070 2023-10-19T10:37:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.505+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5015b9a0 con 0x7f7d500b4070 2023-10-19T10:37:56.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.506+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d500d09b0 con 0x7f7d500b4070 2023-10-19T10:37:56.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.507+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d4c0061a0 con 0x7f7d500b4070 2023-10-19T10:37:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.507+0000 7f7d45ffb640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7d3406a080 0x7f7d3406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.507+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7d4c0aa7a0 con 0x7f7d500b4070 2023-10-19T10:37:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.507+0000 7f7d54ee4640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7d3406a080 0x7f7d3406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.508+0000 7f7d54ee4640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7d3406a080 0x7f7d3406c540 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d5015de10 tx=0x7f7d40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.512+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7d4c075c80 con 0x7f7d500b4070 2023-10-19T10:37:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.685+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d5004e420 con 0x7f7d500b4070 2023-10-19T10:37:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.686+0000 7f7d45ffb640 1 -- 172.21.15.50:0/3653907266 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d4c0792a0 con 0x7f7d500b4070 2023-10-19T10:37:56.689 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:56.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7d3406a080 msgr2=0x7f7d3406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7d3406a080 0x7f7d3406c540 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d5015de10 tx=0x7f7d40008040 comp rx=0 tx=0).stop 2023-10-19T10:37:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 msgr2=0x7f7d50155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d50155e60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009970 tx=0x7f7d4c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 shutdown_connections 2023-10-19T10:37:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7d3406a080 0x7f7d3406c540 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d5015e5e0 0x7f7d5015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5015c960 0x7f7d5015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 --2- 172.21.15.50:0/3653907266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d500b4070 0x7f7d50155e60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 >> 172.21.15.50:0/3653907266 conn(0x7f7d5009b970 msgr2=0x7f7d500b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 shutdown_connections 2023-10-19T10:37:56.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.688+0000 7f7d57949640 1 -- 172.21.15.50:0/3653907266 wait complete. 2023-10-19T10:37:56.701 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:56.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.807+0000 7f8c9c49d640 1 Processor -- start 2023-10-19T10:37:56.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.807+0000 7f8c9c49d640 1 -- start start 2023-10-19T10:37:56.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c9c49d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c940ce8a0 0x7f8c940cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c9c49d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c940d1010 0x7f8c940cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c9c49d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940cec80 con 0x7f8c9415d060 2023-10-19T10:37:56.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c9c49d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c940d0b80 con 0x7f8c940d1010 2023-10-19T10:37:56.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c9c49d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9415dd90 con 0x7f8c940ce8a0 2023-10-19T10:37:56.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c99a38640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c940d1010 0x7f8c940cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c99a38640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c940d1010 0x7f8c940cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55264/0 (socket says 172.21.15.50:55264) 2023-10-19T10:37:56.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c99a38640 1 -- 172.21.15.50:0/3599539720 learned_addr learned my addr 172.21.15.50:0/3599539720 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:56.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.808+0000 7f8c9a239640 1 --2- 172.21.15.50:0/3599539720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c940ce8a0 0x7f8c940cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.809+0000 7f8c99a38640 1 -- 172.21.15.50:0/3599539720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c940ce8a0 msgr2=0x7f8c940cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:56.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.809+0000 7f8c99a38640 1 --2- 172.21.15.50:0/3599539720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c940ce8a0 0x7f8c940cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.809+0000 7f8c99a38640 1 -- 172.21.15.50:0/3599539720 >> v1:172.21.15.50:6789/0 conn(0x7f8c9415d060 legacy=0x7f8c9415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.809+0000 7f8c99a38640 1 -- 172.21.15.50:0/3599539720 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940d0250 con 0x7f8c940d1010 2023-10-19T10:37:56.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.809+0000 7f8c99a38640 1 --2- 172.21.15.50:0/3599539720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940d1010 0x7f8c940cd3d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009870 tx=0x7f8c8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b050859a2bb37db server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c99237640 1 -- 172.21.15.50:0/3599539720 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84031640 con 0x7f8c940d1010 2023-10-19T10:37:56.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c99237640 1 -- 172.21.15.50:0/3599539720 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c840317a0 con 0x7f8c940d1010 2023-10-19T10:37:56.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c99237640 1 -- 172.21.15.50:0/3599539720 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84031b30 con 0x7f8c940d1010 2023-10-19T10:37:56.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3599539720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940d1010 msgr2=0x7f8c940cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3599539720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940d1010 0x7f8c940cd3d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009870 tx=0x7f8c8402b8f0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3599539720 shutdown_connections 2023-10-19T10:37:56.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3599539720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940d1010 0x7f8c940cd3d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3599539720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c940ce8a0 0x7f8c940cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3599539720 >> 172.21.15.50:0/3599539720 conn(0x7f8c94069800 msgr2=0x7f8c94069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:56.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3599539720 shutdown_connections 2023-10-19T10:37:56.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.810+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3599539720 wait complete. 2023-10-19T10:37:56.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.811+0000 7f8c9c49d640 1 Processor -- start 2023-10-19T10:37:56.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.811+0000 7f8c9c49d640 1 -- start start 2023-10-19T10:37:56.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9c49d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 0x7f8c9414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9c49d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c940d1010 0x7f8c9415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.827 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9c49d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c9415d060 0x7f8c9415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9c49d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c5a30 con 0x7f8c940ce8a0 2023-10-19T10:37:56.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9c49d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c4e20 con 0x7f8c940d1010 2023-10-19T10:37:56.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9c49d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c5100 con 0x7f8c9415d060 2023-10-19T10:37:56.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9a239640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 0x7f8c9414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9a239640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 0x7f8c9414b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55272/0 (socket says 172.21.15.50:55272) 2023-10-19T10:37:56.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.812+0000 7f8c9a239640 1 -- 172.21.15.50:0/3950461773 learned_addr learned my addr 172.21.15.50:0/3950461773 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:56.829 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9a239640 1 -- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c9415d060 msgr2=0x7f8c9415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9a239640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c9415d060 0x7f8c9415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9a239640 1 -- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c940d1010 msgr2=0x7f8c9415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9a239640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c940d1010 0x7f8c9415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:56.830 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9a239640 1 -- 172.21.15.50:0/3950461773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840094d0 con 0x7f8c940ce8a0 2023-10-19T10:37:56.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9a239640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 0x7f8c9414b510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e9b0 tx=0x7f8c8c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c009670 con 0x7f8c940ce8a0 2023-10-19T10:37:56.831 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00c9a0 con 0x7f8c940ce8a0 2023-10-19T10:37:56.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c940c53e0 con 0x7f8c940ce8a0 2023-10-19T10:37:56.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00cd50 con 0x7f8c940ce8a0 2023-10-19T10:37:56.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.813+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c940c4aa0 con 0x7f8c940ce8a0 2023-10-19T10:37:56.832 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.814+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c940c8060 con 0x7f8c940ce8a0 2023-10-19T10:37:56.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.814+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8c8c013470 con 0x7f8c940ce8a0 2023-10-19T10:37:56.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.815+0000 7f8c8affd640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c7806a050 0x7f8c7806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:56.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.815+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8c8c088ae0 con 0x7f8c940ce8a0 2023-10-19T10:37:56.833 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.815+0000 7f8c99a38640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c7806a050 0x7f8c7806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.816+0000 7f8c99a38640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c7806a050 0x7f8c7806c510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c84036000 tx=0x7f8c84030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:56.835 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.819+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f8c8c053f30 con 0x7f8c940ce8a0 2023-10-19T10:37:56.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.994+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8c9404e420 con 0x7f8c940ce8a0 2023-10-19T10:37:56.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.994+0000 7f8c8affd640 1 -- 172.21.15.50:0/3950461773 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c8c057550 con 0x7f8c940ce8a0 2023-10-19T10:37:56.997 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:56.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c7806a050 msgr2=0x7f8c7806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c7806a050 0x7f8c7806c510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c84036000 tx=0x7f8c84030000 comp rx=0 tx=0).stop 2023-10-19T10:37:56.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 msgr2=0x7f8c9414b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:56.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 0x7f8c9414b510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e9b0 tx=0x7f8c8c00ee70 comp rx=0 tx=0).stop 2023-10-19T10:37:56.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 shutdown_connections 2023-10-19T10:37:57.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c7806a050 0x7f8c7806c510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c9415d060 0x7f8c9415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c940d1010 0x7f8c9415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 --2- 172.21.15.50:0/3950461773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c940ce8a0 0x7f8c9414b510 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 >> 172.21.15.50:0/3950461773 conn(0x7f8c94069800 msgr2=0x7f8c940cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 shutdown_connections 2023-10-19T10:37:57.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:56.996+0000 7f8c9c49d640 1 -- 172.21.15.50:0/3950461773 wait complete. 2023-10-19T10:37:57.010 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:57.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.117+0000 7fad2f848640 1 Processor -- start 2023-10-19T10:37:57.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.117+0000 7fad2f848640 1 -- start start 2023-10-19T10:37:57.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2f848640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fad280b38f0 0x7fad280d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2f848640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fad280b3090 0x7fad28160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2f848640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad28161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2f848640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280b0190 con 0x7fad280b3090 2023-10-19T10:37:57.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2f848640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280af250 con 0x7fad280b38f0 2023-10-19T10:37:57.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2f848640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280af510 con 0x7fad280b3cd0 2023-10-19T10:37:57.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2cde3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fad280b3090 0x7fad28160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2cde3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fad280b3090 0x7fad28160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55276/0 (socket says 172.21.15.50:55276) 2023-10-19T10:37:57.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.118+0000 7fad2cde3640 1 -- 172.21.15.50:0/3055663192 learned_addr learned my addr 172.21.15.50:0/3055663192 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:57.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2d5e4640 1 --2- 172.21.15.50:0/3055663192 >> v2:172.21.15.183:3300/0 conn(0x7fad280b38f0 0x7fad280d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2cde3640 1 -- 172.21.15.50:0/3055663192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 msgr2=0x7fad28161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:57.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2dde5640 1 --2- 172.21.15.50:0/3055663192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad28161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2cde3640 1 --2- 172.21.15.50:0/3055663192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad28161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2cde3640 1 -- 172.21.15.50:0/3055663192 >> v2:172.21.15.183:3300/0 conn(0x7fad280b38f0 msgr2=0x7fad280d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:57.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2cde3640 1 --2- 172.21.15.50:0/3055663192 >> v2:172.21.15.183:3300/0 conn(0x7fad280b38f0 0x7fad280d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2cde3640 1 -- 172.21.15.50:0/3055663192 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280af7f0 con 0x7fad280b3090 2023-10-19T10:37:57.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.119+0000 7fad2d5e4640 1 --2- 172.21.15.50:0/3055663192 >> v2:172.21.15.183:3300/0 conn(0x7fad280b38f0 0x7fad280d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:57.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2cde3640 1 --2- 172.21.15.50:0/3055663192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28160b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad2808b230 tx=0x7fad1c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4db9d87291a64ba server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad27fff640 1 -- 172.21.15.50:0/3055663192 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c031860 con 0x7fad280b3090 2023-10-19T10:37:57.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad27fff640 1 -- 172.21.15.50:0/3055663192 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c0319c0 con 0x7fad280b3090 2023-10-19T10:37:57.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad27fff640 1 -- 172.21.15.50:0/3055663192 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c031d50 con 0x7fad280b3090 2023-10-19T10:37:57.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 -- 172.21.15.50:0/3055663192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 msgr2=0x7fad28160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 --2- 172.21.15.50:0/3055663192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28160b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad2808b230 tx=0x7fad1c02b900 comp rx=0 tx=0).stop 2023-10-19T10:37:57.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 -- 172.21.15.50:0/3055663192 shutdown_connections 2023-10-19T10:37:57.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 --2- 172.21.15.50:0/3055663192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad28161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 --2- 172.21.15.50:0/3055663192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28160b00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 --2- 172.21.15.50:0/3055663192 >> v2:172.21.15.183:3300/0 conn(0x7fad280b38f0 0x7fad280d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 -- 172.21.15.50:0/3055663192 >> 172.21.15.50:0/3055663192 conn(0x7fad2809b960 msgr2=0x7fad280d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 -- 172.21.15.50:0/3055663192 shutdown_connections 2023-10-19T10:37:57.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.120+0000 7fad2f848640 1 -- 172.21.15.50:0/3055663192 wait complete. 2023-10-19T10:37:57.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.121+0000 7fad2f848640 1 Processor -- start 2023-10-19T10:37:57.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 -- start start 2023-10-19T10:37:57.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad280b38f0 0x7fad280cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad280b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281593f0 con 0x7fad280b3090 2023-10-19T10:37:57.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cb130 con 0x7fad280b38f0 2023-10-19T10:37:57.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2f848640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cb3d0 con 0x7fad280b3cd0 2023-10-19T10:37:57.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2d5e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2d5e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55280/0 (socket says 172.21.15.50:55280) 2023-10-19T10:37:57.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.122+0000 7fad2d5e4640 1 -- 172.21.15.50:0/2573450665 learned_addr learned my addr 172.21.15.50:0/2573450665 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:57.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2d5e4640 1 -- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 msgr2=0x7fad280b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2d5e4640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad280b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2cde3640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad280b38f0 0x7fad280cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2d5e4640 1 -- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad280b38f0 msgr2=0x7fad280cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2d5e4640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad280b38f0 0x7fad280cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2d5e4640 1 -- 172.21.15.50:0/2573450665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0094d0 con 0x7fad280b3090 2023-10-19T10:37:57.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2cde3640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad280b38f0 0x7fad280cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:57.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2d5e4640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28158eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c7c0 tx=0x7fad2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20005b90 con 0x7fad280b3090 2023-10-19T10:37:57.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20005cf0 con 0x7fad280b3090 2023-10-19T10:37:57.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20006250 con 0x7fad280b3090 2023-10-19T10:37:57.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad280cb6b0 con 0x7fad280b3090 2023-10-19T10:37:57.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.123+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad280cbc70 con 0x7fad280b3090 2023-10-19T10:37:57.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.124+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2804e420 con 0x7fad280b3090 2023-10-19T10:37:57.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.124+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fad200063b0 con 0x7fad280b3090 2023-10-19T10:37:57.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.125+0000 7fad25ffb640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad1406a050 0x7fad1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.125+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fad200887c0 con 0x7fad280b3090 2023-10-19T10:37:57.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.125+0000 7fad2cde3640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad1406a050 0x7fad1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.126+0000 7fad2cde3640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad1406a050 0x7fad1406c510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad1c036000 tx=0x7fad1c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.129+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fad20053c10 con 0x7fad280b3090 2023-10-19T10:37:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.305+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad280afaa0 con 0x7fad280b3090 2023-10-19T10:37:57.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.305+0000 7fad25ffb640 1 -- 172.21.15.50:0/2573450665 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad20057230 con 0x7fad280b3090 2023-10-19T10:37:57.308 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:57.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad1406a050 msgr2=0x7fad1406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad1406a050 0x7fad1406c510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad1c036000 tx=0x7fad1c03c040 comp rx=0 tx=0).stop 2023-10-19T10:37:57.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 msgr2=0x7fad28158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28158eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c7c0 tx=0x7fad2000cc80 comp rx=0 tx=0).stop 2023-10-19T10:37:57.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 shutdown_connections 2023-10-19T10:37:57.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fad1406a050 0x7fad1406c510 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad280b3cd0 0x7fad280b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad280b38f0 0x7fad280cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 --2- 172.21.15.50:0/2573450665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad280b3090 0x7fad28158eb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 >> 172.21.15.50:0/2573450665 conn(0x7fad2809b960 msgr2=0x7fad280c7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 shutdown_connections 2023-10-19T10:37:57.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.307+0000 7fad2f848640 1 -- 172.21.15.50:0/2573450665 wait complete. 2023-10-19T10:37:57.320 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:57.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.427+0000 7ff28ad11640 1 Processor -- start 2023-10-19T10:37:57.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.427+0000 7ff28ad11640 1 -- start start 2023-10-19T10:37:57.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff28ad11640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2840b3090 0x7ff2840d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff28ad11640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff284161210 0x7ff2840b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff28ad11640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2840af250 con 0x7ff2840b38f0 2023-10-19T10:37:57.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff28ad11640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2840b0190 con 0x7ff2840b3090 2023-10-19T10:37:57.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff28ad11640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2840af510 con 0x7ff284161210 2023-10-19T10:37:57.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff288aad640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff2840b38f0 0x7ff284160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46734/0 (socket says 172.21.15.50:46734) 2023-10-19T10:37:57.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff288aad640 1 -- 172.21.15.50:0/342438895 learned_addr learned my addr 172.21.15.50:0/342438895 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:57.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.428+0000 7ff283fff640 1 --2- 172.21.15.50:0/342438895 >> v2:172.21.15.50:3300/0 conn(0x7ff2840b3090 0x7ff2840d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.429+0000 7ff2892ae640 1 --2- 172.21.15.50:0/342438895 >> v2:172.21.15.183:3300/0 conn(0x7ff284161210 0x7ff2840b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.429+0000 7ff283fff640 1 -- 172.21.15.50:0/342438895 >> v2:172.21.15.183:3300/0 conn(0x7ff284161210 msgr2=0x7ff2840b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:57.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.429+0000 7ff283fff640 1 --2- 172.21.15.50:0/342438895 >> v2:172.21.15.183:3300/0 conn(0x7ff284161210 0x7ff2840b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.429+0000 7ff283fff640 1 -- 172.21.15.50:0/342438895 >> v1:172.21.15.50:6789/0 conn(0x7ff2840b38f0 legacy=0x7ff284160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.429+0000 7ff283fff640 1 -- 172.21.15.50:0/342438895 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840af7f0 con 0x7ff2840b3090 2023-10-19T10:37:57.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.429+0000 7ff283fff640 1 --2- 172.21.15.50:0/342438895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2840b3090 0x7ff2840d30e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2840b3470 tx=0x7ff27802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f6481f163fc61 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff2837fe640 1 -- 172.21.15.50:0/342438895 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278031830 con 0x7ff2840b3090 2023-10-19T10:37:57.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff2837fe640 1 -- 172.21.15.50:0/342438895 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278031990 con 0x7ff2840b3090 2023-10-19T10:37:57.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff2837fe640 1 -- 172.21.15.50:0/342438895 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278031d20 con 0x7ff2840b3090 2023-10-19T10:37:57.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 -- 172.21.15.50:0/342438895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2840b3090 msgr2=0x7ff2840d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 --2- 172.21.15.50:0/342438895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2840b3090 0x7ff2840d30e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2840b3470 tx=0x7ff27802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 -- 172.21.15.50:0/342438895 shutdown_connections 2023-10-19T10:37:57.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 --2- 172.21.15.50:0/342438895 >> v2:172.21.15.183:3300/0 conn(0x7ff284161210 0x7ff2840b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 --2- 172.21.15.50:0/342438895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2840b3090 0x7ff2840d30e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 -- 172.21.15.50:0/342438895 >> 172.21.15.50:0/342438895 conn(0x7ff28409b960 msgr2=0x7ff2840d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 -- 172.21.15.50:0/342438895 shutdown_connections 2023-10-19T10:37:57.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.430+0000 7ff28ad11640 1 -- 172.21.15.50:0/342438895 wait complete. 2023-10-19T10:37:57.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.431+0000 7ff28ad11640 1 Processor -- start 2023-10-19T10:37:57.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.431+0000 7ff28ad11640 1 -- start start 2023-10-19T10:37:57.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff28ad11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2840b3090 0x7ff284158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff28ad11640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2840b38f0 0x7ff284157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff28ad11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 0x7ff2840cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff28ad11640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff284159c90 con 0x7ff284161210 2023-10-19T10:37:57.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff28ad11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff284159080 con 0x7ff2840b3090 2023-10-19T10:37:57.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff28ad11640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff284159360 con 0x7ff2840b38f0 2023-10-19T10:37:57.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff2892ae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 0x7ff2840cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff2892ae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 0x7ff2840cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55300/0 (socket says 172.21.15.50:55300) 2023-10-19T10:37:57.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff2892ae640 1 -- 172.21.15.50:0/2265496946 learned_addr learned my addr 172.21.15.50:0/2265496946 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:57.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.432+0000 7ff288aad640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2840b3090 0x7ff284158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff283fff640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2840b38f0 0x7ff284157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2892ae640 1 -- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2840b38f0 msgr2=0x7ff284157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2892ae640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2840b38f0 0x7ff284157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2892ae640 1 -- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2840b3090 msgr2=0x7ff284158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2892ae640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2840b3090 0x7ff284158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2892ae640 1 -- 172.21.15.50:0/2265496946 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009400 con 0x7ff284161210 2023-10-19T10:37:57.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2892ae640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 0x7ff2840cb630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff27400c5c0 tx=0x7ff27400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274004220 con 0x7ff284161210 2023-10-19T10:37:57.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff274004380 con 0x7ff284161210 2023-10-19T10:37:57.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff284159640 con 0x7ff284161210 2023-10-19T10:37:57.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274003830 con 0x7ff284161210 2023-10-19T10:37:57.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.433+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2840cbb70 con 0x7ff284161210 2023-10-19T10:37:57.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.434+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28404e420 con 0x7ff284161210 2023-10-19T10:37:57.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.434+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff274003990 con 0x7ff284161210 2023-10-19T10:37:57.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.435+0000 7ff2817fa640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27006a050 0x7ff27006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.435+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff274088520 con 0x7ff284161210 2023-10-19T10:37:57.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.435+0000 7ff288aad640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27006a050 0x7ff27006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.436+0000 7ff288aad640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27006a050 0x7ff27006c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff27c001460 tx=0x7ff27c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.439+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff274053a60 con 0x7ff284161210 2023-10-19T10:37:57.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.616+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff284159dd0 con 0x7ff284161210 2023-10-19T10:37:57.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.616+0000 7ff2817fa640 1 -- 172.21.15.50:0/2265496946 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff274057080 con 0x7ff284161210 2023-10-19T10:37:57.619 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27006a050 msgr2=0x7ff27006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27006a050 0x7ff27006c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff27c001460 tx=0x7ff27c009040 comp rx=0 tx=0).stop 2023-10-19T10:37:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 msgr2=0x7ff2840cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 0x7ff2840cb630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff27400c5c0 tx=0x7ff27400ca80 comp rx=0 tx=0).stop 2023-10-19T10:37:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 shutdown_connections 2023-10-19T10:37:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27006a050 0x7ff27006c510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff284161210 0x7ff2840cb630 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2840b38f0 0x7ff284157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 --2- 172.21.15.50:0/2265496946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2840b3090 0x7ff284158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 >> 172.21.15.50:0/2265496946 conn(0x7ff28409b960 msgr2=0x7ff2840cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 shutdown_connections 2023-10-19T10:37:57.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.618+0000 7ff28ad11640 1 -- 172.21.15.50:0/2265496946 wait complete. 2023-10-19T10:37:57.632 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:37:57.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.738+0000 7f49dcbd2640 1 Processor -- start 2023-10-19T10:37:57.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.738+0000 7f49dcbd2640 1 -- start start 2023-10-19T10:37:57.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49dcbd2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d80b4070 0x7f49d80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49dcbd2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f49d815cdc0 0x7f49d80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49dcbd2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80d3480 con 0x7f49d815c960 2023-10-19T10:37:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49dcbd2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80b4990 con 0x7f49d815cdc0 2023-10-19T10:37:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49dcbd2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d815d280 con 0x7f49d80b4070 2023-10-19T10:37:57.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f49d815cdc0 0x7f49d80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f49d815cdc0 0x7f49d80cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55316/0 (socket says 172.21.15.50:55316) 2023-10-19T10:37:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 -- 172.21.15.50:0/808576731 learned_addr learned my addr 172.21.15.50:0/808576731 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:57.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 -- 172.21.15.50:0/808576731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d80b4070 msgr2=0x7f49d80b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:57.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d659c640 1 --2- 172.21.15.50:0/808576731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d80b4070 0x7f49d80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 --2- 172.21.15.50:0/808576731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d80b4070 0x7f49d80b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 -- 172.21.15.50:0/808576731 >> v1:172.21.15.50:6789/0 conn(0x7f49d815c960 legacy=0x7f49d80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d5d9b640 1 -- 172.21.15.50:0/808576731 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642456875 0 0) 0x7f49d80d3480 con 0x7f49d815c960 2023-10-19T10:37:57.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.739+0000 7f49d6d9d640 1 -- 172.21.15.50:0/808576731 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d815d560 con 0x7f49d815cdc0 2023-10-19T10:37:57.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.740+0000 7f49d6d9d640 1 --2- 172.21.15.50:0/808576731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80cd4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009e50 tx=0x7f49d002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a8f6ed0d3363e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.740+0000 7f49d5d9b640 1 -- 172.21.15.50:0/808576731 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d002cb00 con 0x7f49d815cdc0 2023-10-19T10:37:57.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.740+0000 7f49d5d9b640 1 -- 172.21.15.50:0/808576731 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d002cc60 con 0x7f49d815cdc0 2023-10-19T10:37:57.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.740+0000 7f49d5d9b640 1 -- 172.21.15.50:0/808576731 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00343f0 con 0x7f49d815cdc0 2023-10-19T10:37:57.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 -- 172.21.15.50:0/808576731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 msgr2=0x7f49d80cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/808576731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80cd4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009e50 tx=0x7f49d002bf60 comp rx=0 tx=0).stop 2023-10-19T10:37:57.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 -- 172.21.15.50:0/808576731 shutdown_connections 2023-10-19T10:37:57.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/808576731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80cd4b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/808576731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d80b4070 0x7f49d80b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 -- 172.21.15.50:0/808576731 >> 172.21.15.50:0/808576731 conn(0x7f49d809b970 msgr2=0x7f49d80d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 -- 172.21.15.50:0/808576731 shutdown_connections 2023-10-19T10:37:57.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.741+0000 7f49dcbd2640 1 -- 172.21.15.50:0/808576731 wait complete. 2023-10-19T10:37:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.742+0000 7f49dcbd2640 1 Processor -- start 2023-10-19T10:37:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.742+0000 7f49dcbd2640 1 -- start start 2023-10-19T10:37:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d80b4070 0x7f49d80c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d815c960 0x7f49d81f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80c91b0 con 0x7f49d815cdc0 2023-10-19T10:37:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b5b20 con 0x7f49d80b4070 2023-10-19T10:37:57.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b5210 con 0x7f49d815c960 2023-10-19T10:37:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55332/0 (socket says 172.21.15.50:55332) 2023-10-19T10:37:57.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 -- 172.21.15.50:0/1697818804 learned_addr learned my addr 172.21.15.50:0/1697818804 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 -- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d815c960 msgr2=0x7f49d81f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d815c960 0x7f49d81f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 -- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d80b4070 msgr2=0x7f49d80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d80b4070 0x7f49d80c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 -- 172.21.15.50:0/1697818804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009b30 con 0x7f49d815cdc0 2023-10-19T10:37:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49d6d9d640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80b1180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009e50 tx=0x7f49d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0009510 con 0x7f49d815cdc0 2023-10-19T10:37:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d003e450 con 0x7f49d815cdc0 2023-10-19T10:37:57.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.743+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d815f020 con 0x7f49d815cdc0 2023-10-19T10:37:57.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.744+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d003e800 con 0x7f49d815cdc0 2023-10-19T10:37:57.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.744+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d80b57b0 con 0x7f49d815cdc0 2023-10-19T10:37:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.745+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d804e420 con 0x7f49d815cdc0 2023-10-19T10:37:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.745+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f49d003e960 con 0x7f49d815cdc0 2023-10-19T10:37:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.745+0000 7f49ceffd640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f49b806a080 0x7f49b806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.745+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f49d00ab7a0 con 0x7f49d815cdc0 2023-10-19T10:37:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.746+0000 7f49d659c640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f49b806a080 0x7f49b806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.746+0000 7f49d659c640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f49b806a080 0x7f49b806c540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49c800a890 tx=0x7f49c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.750+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f49d0076c90 con 0x7f49d815cdc0 2023-10-19T10:37:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.923+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f49d80b5c60 con 0x7f49d815cdc0 2023-10-19T10:37:57.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.923+0000 7f49ceffd640 1 -- 172.21.15.50:0/1697818804 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f49d007a2b0 con 0x7f49d815cdc0 2023-10-19T10:37:57.926 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f49b806a080 msgr2=0x7f49b806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f49b806a080 0x7f49b806c540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49c800a890 tx=0x7f49c8008040 comp rx=0 tx=0).stop 2023-10-19T10:37:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 msgr2=0x7f49d80b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80b1180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009e50 tx=0x7f49d0004a40 comp rx=0 tx=0).stop 2023-10-19T10:37:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 shutdown_connections 2023-10-19T10:37:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f49b806a080 0x7f49b806c540 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d815cdc0 0x7f49d80b1180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d815c960 0x7f49d81f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 --2- 172.21.15.50:0/1697818804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d80b4070 0x7f49d80c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 >> 172.21.15.50:0/1697818804 conn(0x7f49d809b970 msgr2=0x7f49d80cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 shutdown_connections 2023-10-19T10:37:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:57.925+0000 7f49dcbd2640 1 -- 172.21.15.50:0/1697818804 wait complete. 2023-10-19T10:37:57.939 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:37:58.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.044+0000 7fba56fa0640 1 Processor -- start 2023-10-19T10:37:58.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.044+0000 7fba56fa0640 1 -- start start 2023-10-19T10:37:58.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba56fa0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fba5015c290 0x7fba500afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba56fa0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fba500cd9d0 0x7fba50070e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba56fa0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba500ca680 con 0x7fba5015d250 2023-10-19T10:37:58.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba56fa0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5015d7a0 con 0x7fba500cd9d0 2023-10-19T10:37:58.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba56fa0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5015c670 con 0x7fba5015c290 2023-10-19T10:37:58.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba5679f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fba500cd9d0 0x7fba50070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba5679f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fba500cd9d0 0x7fba50070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55348/0 (socket says 172.21.15.50:55348) 2023-10-19T10:37:58.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.045+0000 7fba5679f640 1 -- 172.21.15.50:0/2462723302 learned_addr learned my addr 172.21.15.50:0/2462723302 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:58.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba55f9e640 1 --2- 172.21.15.50:0/2462723302 >> v2:172.21.15.183:3300/0 conn(0x7fba5015c290 0x7fba500afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba5679f640 1 -- 172.21.15.50:0/2462723302 >> v2:172.21.15.183:3300/0 conn(0x7fba5015c290 msgr2=0x7fba500afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:58.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba5679f640 1 --2- 172.21.15.50:0/2462723302 >> v2:172.21.15.183:3300/0 conn(0x7fba5015c290 0x7fba500afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba5679f640 1 -- 172.21.15.50:0/2462723302 >> v1:172.21.15.50:6789/0 conn(0x7fba5015d250 legacy=0x7fba50070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba5679f640 1 -- 172.21.15.50:0/2462723302 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500c8950 con 0x7fba500cd9d0 2023-10-19T10:37:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba5679f640 1 --2- 172.21.15.50:0/2462723302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba500cd9d0 0x7fba50070e90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fba500b08a0 tx=0x7fba4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f7fafdb720f91b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.046+0000 7fba54f9c640 1 -- 172.21.15.50:0/2462723302 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48030860 con 0x7fba500cd9d0 2023-10-19T10:37:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba54f9c640 1 -- 172.21.15.50:0/2462723302 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba480309c0 con 0x7fba500cd9d0 2023-10-19T10:37:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba54f9c640 1 -- 172.21.15.50:0/2462723302 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48030d50 con 0x7fba500cd9d0 2023-10-19T10:37:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 -- 172.21.15.50:0/2462723302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba500cd9d0 msgr2=0x7fba50070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 --2- 172.21.15.50:0/2462723302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba500cd9d0 0x7fba50070e90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fba500b08a0 tx=0x7fba4802b9a0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 -- 172.21.15.50:0/2462723302 shutdown_connections 2023-10-19T10:37:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 --2- 172.21.15.50:0/2462723302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba500cd9d0 0x7fba50070e90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 --2- 172.21.15.50:0/2462723302 >> v2:172.21.15.183:3300/0 conn(0x7fba5015c290 0x7fba500afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 -- 172.21.15.50:0/2462723302 >> 172.21.15.50:0/2462723302 conn(0x7fba500cabe0 msgr2=0x7fba500cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 -- 172.21.15.50:0/2462723302 shutdown_connections 2023-10-19T10:37:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.047+0000 7fba56fa0640 1 -- 172.21.15.50:0/2462723302 wait complete. 2023-10-19T10:37:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.048+0000 7fba56fa0640 1 Processor -- start 2023-10-19T10:37:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.048+0000 7fba56fa0640 1 -- start start 2023-10-19T10:37:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba56fa0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba500cd9d0 0x7fba50098e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba56fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba5015c290 0x7fba5009b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba56fa0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 0x7fba500c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba56fa0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015db10 con 0x7fba5015d250 2023-10-19T10:37:58.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba56fa0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015dcf0 con 0x7fba500cd9d0 2023-10-19T10:37:58.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba56fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5015dfd0 con 0x7fba5015c290 2023-10-19T10:37:58.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 0x7fba500c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 0x7fba500c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55358/0 (socket says 172.21.15.50:55358) 2023-10-19T10:37:58.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 -- 172.21.15.50:0/3856212477 learned_addr learned my addr 172.21.15.50:0/3856212477 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:58.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 -- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba5015c290 msgr2=0x7fba5009b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5579d640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba5015c290 0x7fba5009b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba5015c290 0x7fba5009b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 -- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba500cd9d0 msgr2=0x7fba50098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.049+0000 7fba5679f640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba500cd9d0 0x7fba50098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba5679f640 1 -- 172.21.15.50:0/3856212477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480094d0 con 0x7fba5015d250 2023-10-19T10:37:58.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba5679f640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 0x7fba500c4520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fba480090b0 tx=0x7fba48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48039d50 con 0x7fba5015d250 2023-10-19T10:37:58.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba48043400 con 0x7fba5015d250 2023-10-19T10:37:58.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba480437b0 con 0x7fba5015d250 2023-10-19T10:37:58.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5015e2b0 con 0x7fba5015d250 2023-10-19T10:37:58.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.050+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba500c1bc0 con 0x7fba5015d250 2023-10-19T10:37:58.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.051+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5015e770 con 0x7fba5015d250 2023-10-19T10:37:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.051+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fba48043910 con 0x7fba5015d250 2023-10-19T10:37:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.052+0000 7fba4e7fc640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba3c06a050 0x7fba3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.052+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fba4803e070 con 0x7fba5015d250 2023-10-19T10:37:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.052+0000 7fba55f9e640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba3c06a050 0x7fba3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.053+0000 7fba55f9e640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba3c06a050 0x7fba3c06c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba400070b0 tx=0x7fba40007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.056+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fba48076f60 con 0x7fba5015d250 2023-10-19T10:37:58.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.246+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fba50066e80 con 0x7fba5015d250 2023-10-19T10:37:58.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.246+0000 7fba4e7fc640 1 -- 172.21.15.50:0/3856212477 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fba48034080 con 0x7fba5015d250 2023-10-19T10:37:58.249 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:37:58.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba3c06a050 msgr2=0x7fba3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba3c06a050 0x7fba3c06c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba400070b0 tx=0x7fba40007040 comp rx=0 tx=0).stop 2023-10-19T10:37:58.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 msgr2=0x7fba500c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 0x7fba500c4520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fba480090b0 tx=0x7fba48004730 comp rx=0 tx=0).stop 2023-10-19T10:37:58.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 shutdown_connections 2023-10-19T10:37:58.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba3c06a050 0x7fba3c06c510 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5015d250 0x7fba500c4520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba5015c290 0x7fba5009b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 --2- 172.21.15.50:0/3856212477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba500cd9d0 0x7fba50098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 >> 172.21.15.50:0/3856212477 conn(0x7fba500cabe0 msgr2=0x7fba500ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:58.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 shutdown_connections 2023-10-19T10:37:58.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.248+0000 7fba56fa0640 1 -- 172.21.15.50:0/3856212477 wait complete. 2023-10-19T10:37:58.261 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-19T10:37:58.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:58.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-19T10:37:58.293 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:37:58.294 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:37:58.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:37:58.295 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-19T10:37:58.350 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:37:58.350 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:58.369 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:37:58.370 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:37:58.427 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:37:58.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.531+0000 7fd300e41640 1 Processor -- start 2023-10-19T10:37:58.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.532+0000 7fd300e41640 1 -- start start 2023-10-19T10:37:58.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.532+0000 7fd300e41640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd2fc145980 0x7fd2fc0657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.532+0000 7fd300e41640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd2fc0c9270 0x7fd2fc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.532+0000 7fd300e41640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc06a580 con 0x7fd2fc0cc510 2023-10-19T10:37:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.532+0000 7fd300e41640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc06a860 con 0x7fd2fc0c9270 2023-10-19T10:37:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.532+0000 7fd300e41640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc06ab40 con 0x7fd2fc145980 2023-10-19T10:37:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd2fc0c9270 0x7fd2fc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd2fc0c9270 0x7fd2fc0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55364/0 (socket says 172.21.15.50:55364) 2023-10-19T10:37:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 -- 172.21.15.50:0/2296413325 learned_addr learned my addr 172.21.15.50:0/2296413325 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2f9d9b640 1 --2- 172.21.15.50:0/2296413325 >> v2:172.21.15.183:3300/0 conn(0x7fd2fc145980 0x7fd2fc0657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 -- 172.21.15.50:0/2296413325 >> v2:172.21.15.183:3300/0 conn(0x7fd2fc145980 msgr2=0x7fd2fc0657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 --2- 172.21.15.50:0/2296413325 >> v2:172.21.15.183:3300/0 conn(0x7fd2fc145980 0x7fd2fc0657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 -- 172.21.15.50:0/2296413325 >> v1:172.21.15.50:6789/0 conn(0x7fd2fc0cc510 legacy=0x7fd2fc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2fad9d640 1 -- 172.21.15.50:0/2296413325 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc06ae20 con 0x7fd2fc0c9270 2023-10-19T10:37:58.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.533+0000 7fd2f9d9b640 1 --2- 172.21.15.50:0/2296413325 >> v2:172.21.15.183:3300/0 conn(0x7fd2fc145980 0x7fd2fc0657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:58.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd2fad9d640 1 --2- 172.21.15.50:0/2296413325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c9700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0cf9f0 tx=0x7fd2f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a051d1c83602ad73 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd2f959a640 1 -- 172.21.15.50:0/2296413325 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f402ca80 con 0x7fd2fc0c9270 2023-10-19T10:37:58.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd2f959a640 1 -- 172.21.15.50:0/2296413325 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f402cbe0 con 0x7fd2fc0c9270 2023-10-19T10:37:58.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd2f959a640 1 -- 172.21.15.50:0/2296413325 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40343f0 con 0x7fd2fc0c9270 2023-10-19T10:37:58.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd300e41640 1 -- 172.21.15.50:0/2296413325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0c9270 msgr2=0x7fd2fc0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd300e41640 1 --2- 172.21.15.50:0/2296413325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c9700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0cf9f0 tx=0x7fd2f402bee0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd300e41640 1 -- 172.21.15.50:0/2296413325 shutdown_connections 2023-10-19T10:37:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd300e41640 1 --2- 172.21.15.50:0/2296413325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c9700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd300e41640 1 --2- 172.21.15.50:0/2296413325 >> v2:172.21.15.183:3300/0 conn(0x7fd2fc145980 0x7fd2fc0657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.534+0000 7fd300e41640 1 -- 172.21.15.50:0/2296413325 >> 172.21.15.50:0/2296413325 conn(0x7fd2fc0cb3c0 msgr2=0x7fd2fc0cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.535+0000 7fd300e41640 1 -- 172.21.15.50:0/2296413325 shutdown_connections 2023-10-19T10:37:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.535+0000 7fd300e41640 1 -- 172.21.15.50:0/2296413325 wait complete. 2023-10-19T10:37:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.536+0000 7fd300e41640 1 Processor -- start 2023-10-19T10:37:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.536+0000 7fd300e41640 1 -- start start 2023-10-19T10:37:58.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 0x7fd2fc1f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2fc145980 0x7fd2fc1f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1f2e20 con 0x7fd2fc0cc510 2023-10-19T10:37:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c6560 con 0x7fd2fc0c9270 2023-10-19T10:37:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c6840 con 0x7fd2fc145980 2023-10-19T10:37:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 0x7fd2fc1f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 0x7fd2fc1f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55372/0 (socket says 172.21.15.50:55372) 2023-10-19T10:37:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 -- 172.21.15.50:0/2710830223 learned_addr learned my addr 172.21.15.50:0/2710830223 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2fad9d640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2fc145980 0x7fd2fc1f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2fa59c640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 -- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2fc145980 msgr2=0x7fd2fc1f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2fc145980 0x7fd2fc1f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 -- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2fc0c9270 msgr2=0x7fd2fc0c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 -- 172.21.15.50:0/2710830223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009b30 con 0x7fd2fc0cc510 2023-10-19T10:37:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2fa59c640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:37:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2f9d9b640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 0x7fd2fc1f28e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c7f0 tx=0x7fd2f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000cea0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0007a30 con 0x7fd2fc0cc510 2023-10-19T10:37:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc0c6b20 con 0x7fd2fc0cc510 2023-10-19T10:37:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0007dc0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.537+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc0c70e0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.538+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc06b0e0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.539+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd2f00113f0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.539+0000 7fd2eaffd640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd2dc06a050 0x7fd2dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.540+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd2f0088a70 con 0x7fd2fc0cc510 2023-10-19T10:37:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.540+0000 7fd2fa59c640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd2dc06a050 0x7fd2dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.540+0000 7fd2fa59c640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd2dc06a050 0x7fd2dc06c510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc15eaf0 tx=0x7fd2ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.544+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd2fc06b0e0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.718+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2fc06b0e0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.720+0000 7fd2eaffd640 1 -- 172.21.15.50:0/2710830223 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fd2fc06b0e0 con 0x7fd2fc0cc510 2023-10-19T10:37:58.722 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:37:58.723 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:37:58.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd2dc06a050 msgr2=0x7fd2dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd2dc06a050 0x7fd2dc06c510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc15eaf0 tx=0x7fd2ec009040 comp rx=0 tx=0).stop 2023-10-19T10:37:58.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 msgr2=0x7fd2fc1f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 0x7fd2fc1f28e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c7f0 tx=0x7fd2f000ccb0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 shutdown_connections 2023-10-19T10:37:58.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd2dc06a050 0x7fd2dc06c510 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2fc145980 0x7fd2fc1f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.721+0000 7fd300e41640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2fc0cc510 0x7fd2fc1f28e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.722+0000 7fd300e41640 1 --2- 172.21.15.50:0/2710830223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2fc0c9270 0x7fd2fc0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.722+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 >> 172.21.15.50:0/2710830223 conn(0x7fd2fc0cb3c0 msgr2=0x7fd2fc0ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:58.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.722+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 shutdown_connections 2023-10-19T10:37:58.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.722+0000 7fd300e41640 1 -- 172.21.15.50:0/2710830223 wait complete. 2023-10-19T10:37:58.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:37:58.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:37:58.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.842+0000 7fb94a7f7640 1 Processor -- start 2023-10-19T10:37:58.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.842+0000 7fb94a7f7640 1 -- start start 2023-10-19T10:37:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb94a7f7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb9440ca690 0x7fb9440cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb94a7f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb94415a310 0x7fb94406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb94a7f7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb94406cf00 0x7fb9440c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb94a7f7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94406d370 con 0x7fb9440ca690 2023-10-19T10:37:58.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb94a7f7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440cde00 con 0x7fb94406cf00 2023-10-19T10:37:58.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb94a7f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440ccb00 con 0x7fb94415a310 2023-10-19T10:37:58.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb9440ca690 0x7fb9440cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb9440ca690 0x7fb9440cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55374/0 (socket says 172.21.15.50:55374) 2023-10-19T10:37:58.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 -- 172.21.15.50:0/3739256106 learned_addr learned my addr 172.21.15.50:0/3739256106 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 -- 172.21.15.50:0/3739256106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb94415a310 msgr2=0x7fb94406c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb9437fe640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb94415a310 0x7fb94406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb948d94640 1 --2- 172.21.15.50:0/3739256106 >> v2:172.21.15.183:3300/0 conn(0x7fb94406cf00 0x7fb9440c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb94415a310 0x7fb94406c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 -- 172.21.15.50:0/3739256106 >> v2:172.21.15.183:3300/0 conn(0x7fb94406cf00 msgr2=0x7fb9440c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:58.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 --2- 172.21.15.50:0/3739256106 >> v2:172.21.15.183:3300/0 conn(0x7fb94406cf00 0x7fb9440c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb943fff640 1 -- 172.21.15.50:0/3739256106 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440ccde0 con 0x7fb9440ca690 2023-10-19T10:37:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb9437fe640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb94415a310 0x7fb94406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.843+0000 7fb948d94640 1 --2- 172.21.15.50:0/3739256106 >> v2:172.21.15.183:3300/0 conn(0x7fb94406cf00 0x7fb9440c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.844+0000 7fb943fff640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9440ca690 0x7fb9440cd8c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb944153d90 tx=0x7fb93802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ecb218b78eabb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.844+0000 7fb942ffd640 1 -- 172.21.15.50:0/3739256106 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938031870 con 0x7fb9440ca690 2023-10-19T10:37:58.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.844+0000 7fb942ffd640 1 -- 172.21.15.50:0/3739256106 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9380319d0 con 0x7fb9440ca690 2023-10-19T10:37:58.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.844+0000 7fb942ffd640 1 -- 172.21.15.50:0/3739256106 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938031d60 con 0x7fb9440ca690 2023-10-19T10:37:58.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 -- 172.21.15.50:0/3739256106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9440ca690 msgr2=0x7fb9440cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9440ca690 0x7fb9440cd8c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb944153d90 tx=0x7fb93802b910 comp rx=0 tx=0).stop 2023-10-19T10:37:58.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 -- 172.21.15.50:0/3739256106 shutdown_connections 2023-10-19T10:37:58.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/3739256106 >> v2:172.21.15.183:3300/0 conn(0x7fb94406cf00 0x7fb9440c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb94415a310 0x7fb94406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/3739256106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9440ca690 0x7fb9440cd8c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 -- 172.21.15.50:0/3739256106 >> 172.21.15.50:0/3739256106 conn(0x7fb9440afbe0 msgr2=0x7fb9440affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 -- 172.21.15.50:0/3739256106 shutdown_connections 2023-10-19T10:37:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.845+0000 7fb94a7f7640 1 -- 172.21.15.50:0/3739256106 wait complete. 2023-10-19T10:37:58.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.846+0000 7fb94a7f7640 1 Processor -- start 2023-10-19T10:37:58.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.846+0000 7fb94a7f7640 1 -- start start 2023-10-19T10:37:58.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb94a7f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb94406cf00 0x7fb944156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb94a7f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9440ca690 0x7fb9441f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb94a7f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 0x7fb9441f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb94a7f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94415d060 con 0x7fb94415a310 2023-10-19T10:37:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb94a7f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94415c3e0 con 0x7fb94406cf00 2023-10-19T10:37:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb94a7f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94415c6c0 con 0x7fb9440ca690 2023-10-19T10:37:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 0x7fb9441f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 0x7fb9441f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55390/0 (socket says 172.21.15.50:55390) 2023-10-19T10:37:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 -- 172.21.15.50:0/928178467 learned_addr learned my addr 172.21.15.50:0/928178467 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:58.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 -- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9440ca690 msgr2=0x7fb9441f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb9437fe640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9440ca690 0x7fb9441f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb943fff640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb94406cf00 0x7fb944156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9440ca690 0x7fb9441f5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 -- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb94406cf00 msgr2=0x7fb944156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:58.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb94406cf00 0x7fb944156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:58.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.847+0000 7fb948d94640 1 -- 172.21.15.50:0/928178467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380094d0 con 0x7fb94415a310 2023-10-19T10:37:58.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.848+0000 7fb948d94640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 0x7fb9441f57d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00cd50 tx=0x7fb93c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.848+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00f680 con 0x7fb94415a310 2023-10-19T10:37:58.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.848+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c00f800 con 0x7fb94415a310 2023-10-19T10:37:58.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.848+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94415c9a0 con 0x7fb94415a310 2023-10-19T10:37:58.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.848+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00fbb0 con 0x7fb94415a310 2023-10-19T10:37:58.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.848+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9441f5f10 con 0x7fb94415a310 2023-10-19T10:37:58.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.849+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94415d1a0 con 0x7fb94415a310 2023-10-19T10:37:58.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.849+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb93c00b040 con 0x7fb94415a310 2023-10-19T10:37:58.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.850+0000 7fb940ff9640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb93006a050 0x7fb93006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:58.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.850+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb93c089330 con 0x7fb94415a310 2023-10-19T10:37:58.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.850+0000 7fb943fff640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb93006a050 0x7fb93006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:58.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.850+0000 7fb943fff640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb93006a050 0x7fb93006c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb938036000 tx=0x7fb93803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:58.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:58.854+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb94415d1a0 con 0x7fb94415a310 2023-10-19T10:37:59.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.028+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb94415d1a0 con 0x7fb94415a310 2023-10-19T10:37:59.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.029+0000 7fb940ff9640 1 -- 172.21.15.50:0/928178467 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb94415d1a0 con 0x7fb94415a310 2023-10-19T10:37:59.032 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:37:59.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb93006a050 msgr2=0x7fb93006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb93006a050 0x7fb93006c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb938036000 tx=0x7fb93803c040 comp rx=0 tx=0).stop 2023-10-19T10:37:59.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 msgr2=0x7fb9441f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 0x7fb9441f57d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00cd50 tx=0x7fb93c00a650 comp rx=0 tx=0).stop 2023-10-19T10:37:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 shutdown_connections 2023-10-19T10:37:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb93006a050 0x7fb93006c510 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb94415a310 0x7fb9441f57d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9440ca690 0x7fb9441f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 --2- 172.21.15.50:0/928178467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb94406cf00 0x7fb944156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 >> 172.21.15.50:0/928178467 conn(0x7fb9440afbe0 msgr2=0x7fb9440ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 shutdown_connections 2023-10-19T10:37:59.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.031+0000 7fb94a7f7640 1 -- 172.21.15.50:0/928178467 wait complete. 2023-10-19T10:37:59.044 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:37:59.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.153+0000 7f9106970640 1 Processor -- start 2023-10-19T10:37:59.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.153+0000 7f9106970640 1 -- start start 2023-10-19T10:37:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f9106970640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f91000b4070 0x7f91000b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f9106970640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f910015c960 0x7f91000d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f9106970640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000b4990 con 0x7f910015cdf0 2023-10-19T10:37:59.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f9106970640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910015d280 con 0x7f910015c960 2023-10-19T10:37:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f9106970640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910015d560 con 0x7f91000b4070 2023-10-19T10:37:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f910015c960 0x7f91000d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f910015c960 0x7f91000d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55398/0 (socket says 172.21.15.50:55398) 2023-10-19T10:37:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 -- 172.21.15.50:0/1191377524 learned_addr learned my addr 172.21.15.50:0/1191377524 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:59.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90fffff640 1 --2- 172.21.15.50:0/1191377524 >> v2:172.21.15.183:3300/0 conn(0x7f91000b4070 0x7f91000b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 -- 172.21.15.50:0/1191377524 >> v2:172.21.15.183:3300/0 conn(0x7f91000b4070 msgr2=0x7f91000b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 --2- 172.21.15.50:0/1191377524 >> v2:172.21.15.183:3300/0 conn(0x7f91000b4070 0x7f91000b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 -- 172.21.15.50:0/1191377524 >> v1:172.21.15.50:6789/0 conn(0x7f910015cdf0 legacy=0x7f9100160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.154+0000 7f90ff7fe640 1 -- 172.21.15.50:0/1191377524 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910015d840 con 0x7f910015c960 2023-10-19T10:37:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.155+0000 7f90ff7fe640 1 --2- 172.21.15.50:0/1191377524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910015c960 0x7f91000d2d70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f910008a3b0 tx=0x7f90f002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b542ec39a454559 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.155+0000 7f90feffd640 1 -- 172.21.15.50:0/1191377524 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0030860 con 0x7f910015c960 2023-10-19T10:37:59.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.155+0000 7f90feffd640 1 -- 172.21.15.50:0/1191377524 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f00309c0 con 0x7f910015c960 2023-10-19T10:37:59.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.155+0000 7f90feffd640 1 -- 172.21.15.50:0/1191377524 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0030d50 con 0x7f910015c960 2023-10-19T10:37:59.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 -- 172.21.15.50:0/1191377524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910015c960 msgr2=0x7f91000d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 --2- 172.21.15.50:0/1191377524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910015c960 0x7f91000d2d70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f910008a3b0 tx=0x7f90f002b9a0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 -- 172.21.15.50:0/1191377524 shutdown_connections 2023-10-19T10:37:59.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 --2- 172.21.15.50:0/1191377524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f910015c960 0x7f91000d2d70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 --2- 172.21.15.50:0/1191377524 >> v2:172.21.15.183:3300/0 conn(0x7f91000b4070 0x7f91000b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 -- 172.21.15.50:0/1191377524 >> 172.21.15.50:0/1191377524 conn(0x7f910009b970 msgr2=0x7f91000d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 -- 172.21.15.50:0/1191377524 shutdown_connections 2023-10-19T10:37:59.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.156+0000 7f9106970640 1 -- 172.21.15.50:0/1191377524 wait complete. 2023-10-19T10:37:59.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.157+0000 7f9106970640 1 Processor -- start 2023-10-19T10:37:59.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.157+0000 7f9106970640 1 -- start start 2023-10-19T10:37:59.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9106970640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91000b4070 0x7f91000b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9106970640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f910015c960 0x7f9100158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9106970640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 0x7f91000cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9106970640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000b6df0 con 0x7f91000cbf50 2023-10-19T10:37:59.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9106970640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100159340 con 0x7f910015c960 2023-10-19T10:37:59.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9106970640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000cc940 con 0x7f91000b4070 2023-10-19T10:37:59.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9104f0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 0x7f91000cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9104f0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 0x7f91000cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55404/0 (socket says 172.21.15.50:55404) 2023-10-19T10:37:59.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.158+0000 7f9104f0d640 1 -- 172.21.15.50:0/3363565825 learned_addr learned my addr 172.21.15.50:0/3363565825 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:59.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90ff7fe640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f910015c960 0x7f9100158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90fffff640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91000b4070 0x7f91000b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9104f0d640 1 -- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91000b4070 msgr2=0x7f91000b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9104f0d640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91000b4070 0x7f91000b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9104f0d640 1 -- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f910015c960 msgr2=0x7f9100158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9104f0d640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f910015c960 0x7f9100158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9104f0d640 1 -- 172.21.15.50:0/3363565825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00094d0 con 0x7f91000cbf50 2023-10-19T10:37:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90ff7fe640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f910015c960 0x7f9100158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90fffff640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91000b4070 0x7f91000b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9104f0d640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 0x7f91000cc400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c6b0 tx=0x7f90f800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8003b70 con 0x7f91000cbf50 2023-10-19T10:37:59.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f8003cf0 con 0x7f91000cbf50 2023-10-19T10:37:59.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91000c9e60 con 0x7f91000cbf50 2023-10-19T10:37:59.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8003ea0 con 0x7f91000cbf50 2023-10-19T10:37:59.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.159+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91000ca420 con 0x7f91000cbf50 2023-10-19T10:37:59.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.160+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91000c70e0 con 0x7f91000cbf50 2023-10-19T10:37:59.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.160+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f90f8005310 con 0x7f91000cbf50 2023-10-19T10:37:59.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.161+0000 7f90fcff9640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f90ec06a080 0x7f90ec06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.161+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f90f8088660 con 0x7f91000cbf50 2023-10-19T10:37:59.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.161+0000 7f90fffff640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f90ec06a080 0x7f90ec06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.162+0000 7f90fffff640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f90ec06a080 0x7f90ec06c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91000b7210 tx=0x7f90f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.165+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f91000c70e0 con 0x7f91000cbf50 2023-10-19T10:37:59.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.340+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91000c70e0 con 0x7f91000cbf50 2023-10-19T10:37:59.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.340+0000 7f90fcff9640 1 -- 172.21.15.50:0/3363565825 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91000c70e0 con 0x7f91000cbf50 2023-10-19T10:37:59.343 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:37:59.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f90ec06a080 msgr2=0x7f90ec06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f90ec06a080 0x7f90ec06c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91000b7210 tx=0x7f90f4008040 comp rx=0 tx=0).stop 2023-10-19T10:37:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 msgr2=0x7f91000cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 0x7f91000cc400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c6b0 tx=0x7f90f800cb70 comp rx=0 tx=0).stop 2023-10-19T10:37:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 shutdown_connections 2023-10-19T10:37:59.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f90ec06a080 0x7f90ec06c540 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91000cbf50 0x7f91000cc400 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f910015c960 0x7f9100158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 --2- 172.21.15.50:0/3363565825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f91000b4070 0x7f91000b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 >> 172.21.15.50:0/3363565825 conn(0x7f910009b970 msgr2=0x7f910015e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 shutdown_connections 2023-10-19T10:37:59.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.342+0000 7f9106970640 1 -- 172.21.15.50:0/3363565825 wait complete. 2023-10-19T10:37:59.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:37:59.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 Processor -- start 2023-10-19T10:37:59.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 -- start start 2023-10-19T10:37:59.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b38f0 0x7fc3600d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc3600b3090 0x7fc360160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3600af250 con 0x7fc3600b3cd0 2023-10-19T10:37:59.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3600b0190 con 0x7fc3600b3090 2023-10-19T10:37:59.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.462+0000 7fc36592e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600af510 con 0x7fc3600b38f0 2023-10-19T10:37:59.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc35f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc3600b3cd0 0x7fc3600b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46768/0 (socket says 172.21.15.50:46768) 2023-10-19T10:37:59.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc35f7fe640 1 -- 172.21.15.50:0/2977275357 learned_addr learned my addr 172.21.15.50:0/2977275357 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:59.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc357fff640 1 --2- 172.21.15.50:0/2977275357 >> v2:172.21.15.183:3300/0 conn(0x7fc3600b3090 0x7fc360160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc35effd640 1 --2- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b38f0 0x7fc3600d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc35e7fc640 1 -- 172.21.15.50:0/2977275357 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764533678 0 0) 0x7fc3600af250 con 0x7fc3600b3cd0 2023-10-19T10:37:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc35e7fc640 1 -- 172.21.15.50:0/2977275357 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c003520 con 0x7fc3600b3cd0 2023-10-19T10:37:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc357fff640 1 -- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b38f0 msgr2=0x7fc3600d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc357fff640 1 --2- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b38f0 0x7fc3600d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc357fff640 1 -- 172.21.15.50:0/2977275357 >> v1:172.21.15.50:6789/0 conn(0x7fc3600b3cd0 legacy=0x7fc3600b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.463+0000 7fc357fff640 1 -- 172.21.15.50:0/2977275357 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600af7f0 con 0x7fc3600b3090 2023-10-19T10:37:59.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.464+0000 7fc357fff640 1 --2- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b3090 0x7fc360160b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc3480097a0 tx=0x7fc34802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8cbb2c7077a650 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.464+0000 7fc35e7fc640 1 -- 172.21.15.50:0/2977275357 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348031830 con 0x7fc3600b3090 2023-10-19T10:37:59.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.464+0000 7fc35e7fc640 1 -- 172.21.15.50:0/2977275357 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc348031990 con 0x7fc3600b3090 2023-10-19T10:37:59.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.464+0000 7fc35e7fc640 1 -- 172.21.15.50:0/2977275357 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348031d20 con 0x7fc3600b3090 2023-10-19T10:37:59.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 -- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b3090 msgr2=0x7fc360160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 --2- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b3090 0x7fc360160b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc3480097a0 tx=0x7fc34802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 -- 172.21.15.50:0/2977275357 shutdown_connections 2023-10-19T10:37:59.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 --2- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b3090 0x7fc360160b00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 --2- 172.21.15.50:0/2977275357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b38f0 0x7fc3600d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 -- 172.21.15.50:0/2977275357 >> 172.21.15.50:0/2977275357 conn(0x7fc36009b960 msgr2=0x7fc3600d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 -- 172.21.15.50:0/2977275357 shutdown_connections 2023-10-19T10:37:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.465+0000 7fc36592e640 1 -- 172.21.15.50:0/2977275357 wait complete. 2023-10-19T10:37:59.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.466+0000 7fc36592e640 1 Processor -- start 2023-10-19T10:37:59.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.466+0000 7fc36592e640 1 -- start start 2023-10-19T10:37:59.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc36592e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b3090 0x7fc360156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc36592e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b38f0 0x7fc3600c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc36592e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 0x7fc3600c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc36592e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc360157a60 con 0x7fc3600b3cd0 2023-10-19T10:37:59.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc36592e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600c83a0 con 0x7fc3600b38f0 2023-10-19T10:37:59.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc36592e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36015c950 con 0x7fc3600b3090 2023-10-19T10:37:59.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 0x7fc3600c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 0x7fc3600c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55420/0 (socket says 172.21.15.50:55420) 2023-10-19T10:37:59.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 -- 172.21.15.50:0/118622162 learned_addr learned my addr 172.21.15.50:0/118622162 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc357fff640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b38f0 0x7fc3600c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35effd640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b3090 0x7fc360156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 -- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b3090 msgr2=0x7fc360156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b3090 0x7fc360156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 -- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b38f0 msgr2=0x7fc3600c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b38f0 0x7fc3600c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 -- 172.21.15.50:0/118622162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348009400 con 0x7fc3600b3cd0 2023-10-19T10:37:59.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.467+0000 7fc35f7fe640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 0x7fc3600c8640 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc35800cad0 tx=0x7fc35800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.468+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800ac30 con 0x7fc3600b3cd0 2023-10-19T10:37:59.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.468+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35800ad90 con 0x7fc3600b3cd0 2023-10-19T10:37:59.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.468+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3601571c0 con 0x7fc3600b3cd0 2023-10-19T10:37:59.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.468+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358012620 con 0x7fc3600b3cd0 2023-10-19T10:37:59.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.468+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc360157780 con 0x7fc3600b3cd0 2023-10-19T10:37:59.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.469+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36004e420 con 0x7fc3600b3cd0 2023-10-19T10:37:59.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.469+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc358007070 con 0x7fc3600b3cd0 2023-10-19T10:37:59.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.470+0000 7fc35cff9640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc34006a050 0x7fc34006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.470+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc358088790 con 0x7fc3600b3cd0 2023-10-19T10:37:59.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.470+0000 7fc35effd640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc34006a050 0x7fc34006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.471+0000 7fc35effd640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc34006a050 0x7fc34006c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3600cbcb0 tx=0x7fc350009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.475+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc358053c00 con 0x7fc3600b3cd0 2023-10-19T10:37:59.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.648+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3600b86b0 con 0x7fc3600b3cd0 2023-10-19T10:37:59.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.648+0000 7fc35cff9640 1 -- 172.21.15.50:0/118622162 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc358057220 con 0x7fc3600b3cd0 2023-10-19T10:37:59.651 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:37:59.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc34006a050 msgr2=0x7fc34006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc34006a050 0x7fc34006c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3600cbcb0 tx=0x7fc350009110 comp rx=0 tx=0).stop 2023-10-19T10:37:59.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 msgr2=0x7fc3600c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 0x7fc3600c8640 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc35800cad0 tx=0x7fc35800cf90 comp rx=0 tx=0).stop 2023-10-19T10:37:59.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 shutdown_connections 2023-10-19T10:37:59.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc34006a050 0x7fc34006c510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3600b3cd0 0x7fc3600c8640 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3600b38f0 0x7fc3600c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 --2- 172.21.15.50:0/118622162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3600b3090 0x7fc360156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 >> 172.21.15.50:0/118622162 conn(0x7fc36009b960 msgr2=0x7fc3600d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 shutdown_connections 2023-10-19T10:37:59.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.650+0000 7fc36592e640 1 -- 172.21.15.50:0/118622162 wait complete. 2023-10-19T10:37:59.664 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:37:59.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.770+0000 7f11443cb640 1 Processor -- start 2023-10-19T10:37:59.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.771+0000 7f11443cb640 1 -- start start 2023-10-19T10:37:59.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.771+0000 7f11443cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c15fe20 0x7f113c06b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.771+0000 7f11443cb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f113c06d6e0 0x7f113c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.771+0000 7f11443cb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113c069800 con 0x7f113c0b1100 2023-10-19T10:37:59.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.771+0000 7f11443cb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113c06c810 con 0x7f113c06d6e0 2023-10-19T10:37:59.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.771+0000 7f11443cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c069dc0 con 0x7f113c15fe20 2023-10-19T10:37:59.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1142968640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f113c0b1100 0x7f113c160270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46784/0 (socket says 172.21.15.50:46784) 2023-10-19T10:37:59.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1142968640 1 -- 172.21.15.50:0/1388351405 learned_addr learned my addr 172.21.15.50:0/1388351405 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:37:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1142167640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c15fe20 0x7f113c06b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141966640 1 --2- 172.21.15.50:0/1388351405 >> v2:172.21.15.183:3300/0 conn(0x7f113c06d6e0 0x7f113c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141165640 1 -- 172.21.15.50:0/1388351405 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197006288 0 0) 0x7f113c069800 con 0x7f113c0b1100 2023-10-19T10:37:59.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141165640 1 -- 172.21.15.50:0/1388351405 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1128003520 con 0x7f113c0b1100 2023-10-19T10:37:59.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141966640 1 -- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c15fe20 msgr2=0x7f113c06b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:37:59.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141966640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c15fe20 0x7f113c06b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141966640 1 -- 172.21.15.50:0/1388351405 >> v1:172.21.15.50:6789/0 conn(0x7f113c0b1100 legacy=0x7f113c160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141165640 1 -- 172.21.15.50:0/1388351405 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127900301 0 0) 0x7f1128003520 con 0x7f113c0b1100 2023-10-19T10:37:59.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1141966640 1 -- 172.21.15.50:0/1388351405 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c065640 con 0x7f113c06d6e0 2023-10-19T10:37:59.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.772+0000 7f1142167640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c15fe20 0x7f113c06b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:37:59.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.773+0000 7f1141966640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c06bed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f112c0097a0 tx=0x7f112c02b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9859ed968e9518d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.773+0000 7f1141165640 1 -- 172.21.15.50:0/1388351405 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340019f0 con 0x7f113c06d6e0 2023-10-19T10:37:59.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.773+0000 7f1141165640 1 -- 172.21.15.50:0/1388351405 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c02e050 con 0x7f113c06d6e0 2023-10-19T10:37:59.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.773+0000 7f1141165640 1 -- 172.21.15.50:0/1388351405 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c030710 con 0x7f113c06d6e0 2023-10-19T10:37:59.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 -- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 msgr2=0x7f113c06bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c06bed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f112c0097a0 tx=0x7f112c02b660 comp rx=0 tx=0).stop 2023-10-19T10:37:59.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 -- 172.21.15.50:0/1388351405 shutdown_connections 2023-10-19T10:37:59.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c06bed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 --2- 172.21.15.50:0/1388351405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c15fe20 0x7f113c06b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 -- 172.21.15.50:0/1388351405 >> 172.21.15.50:0/1388351405 conn(0x7f113c0b03f0 msgr2=0x7f113c0b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 -- 172.21.15.50:0/1388351405 shutdown_connections 2023-10-19T10:37:59.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.774+0000 7f11443cb640 1 -- 172.21.15.50:0/1388351405 wait complete. 2023-10-19T10:37:59.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.775+0000 7f11443cb640 1 Processor -- start 2023-10-19T10:37:59.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.775+0000 7f11443cb640 1 -- start start 2023-10-19T10:37:59.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c1f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c0b1100 0x7f113c1f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 0x7f113c1f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0c73f0 con 0x7f113c15fe20 2023-10-19T10:37:59.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0b2bb0 con 0x7f113c06d6e0 2023-10-19T10:37:59.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0c6870 con 0x7f113c0b1100 2023-10-19T10:37:59.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 0x7f113c1f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 0x7f113c1f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55432/0 (socket says 172.21.15.50:55432) 2023-10-19T10:37:59.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 -- 172.21.15.50:0/2493915995 learned_addr learned my addr 172.21.15.50:0/2493915995 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:37:59.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1141966640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c0b1100 0x7f113c1f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142167640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c1f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 -- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c0b1100 msgr2=0x7f113c1f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c0b1100 0x7f113c1f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 -- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 msgr2=0x7f113c1f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c1f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 -- 172.21.15.50:0/2493915995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009400 con 0x7f113c15fe20 2023-10-19T10:37:59.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142167640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c1f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:37:59.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1142968640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 0x7f113c1f5730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f113400cc40 tx=0x7f113400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400ce50 con 0x7f113c15fe20 2023-10-19T10:37:59.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134012400 con 0x7f113c15fe20 2023-10-19T10:37:59.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.776+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c009400 con 0x7f113c15fe20 2023-10-19T10:37:59.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.777+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340127b0 con 0x7f113c15fe20 2023-10-19T10:37:59.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.777+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c0c6e30 con 0x7f113c15fe20 2023-10-19T10:37:59.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.777+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c04e420 con 0x7f113c15fe20 2023-10-19T10:37:59.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.778+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1134012910 con 0x7f113c15fe20 2023-10-19T10:37:59.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.778+0000 7f1132ffd640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f112006a050 0x7f112006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:37:59.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.779+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1134088fd0 con 0x7f113c15fe20 2023-10-19T10:37:59.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.779+0000 7f1142167640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f112006a050 0x7f112006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:37:59.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.779+0000 7f1142167640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f112006a050 0x7f112006c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1138009bb0 tx=0x7f1138008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:37:59.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.783+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f11340543a0 con 0x7f113c15fe20 2023-10-19T10:37:59.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.957+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f113c065890 con 0x7f113c15fe20 2023-10-19T10:37:59.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.958+0000 7f1132ffd640 1 -- 172.21.15.50:0/2493915995 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11340579c0 con 0x7f113c15fe20 2023-10-19T10:37:59.961 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:37:59.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.959+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f112006a050 msgr2=0x7f112006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.959+0000 7f11443cb640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f112006a050 0x7f112006c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1138009bb0 tx=0x7f1138008040 comp rx=0 tx=0).stop 2023-10-19T10:37:59.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.959+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 msgr2=0x7f113c1f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:37:59.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.959+0000 7f11443cb640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 0x7f113c1f5730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f113400cc40 tx=0x7f113400a430 comp rx=0 tx=0).stop 2023-10-19T10:37:59.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 shutdown_connections 2023-10-19T10:37:59.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f112006a050 0x7f112006c510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f113c15fe20 0x7f113c1f5730 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f113c0b1100 0x7f113c1f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 --2- 172.21.15.50:0/2493915995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f113c06d6e0 0x7f113c1f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:37:59.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 >> 172.21.15.50:0/2493915995 conn(0x7f113c0b03f0 msgr2=0x7f113c0c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:37:59.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 shutdown_connections 2023-10-19T10:37:59.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:37:59.960+0000 7f11443cb640 1 -- 172.21.15.50:0/2493915995 wait complete. 2023-10-19T10:37:59.973 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:00.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.079+0000 7f579fbb2640 1 Processor -- start 2023-10-19T10:38:00.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.079+0000 7f579fbb2640 1 -- start start 2023-10-19T10:38:00.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579fbb2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f57980ce730 0x7f579806f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579fbb2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5798145880 0x7f579806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579fbb2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57980701b0 con 0x7f57980b26e0 2023-10-19T10:38:00.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579fbb2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579806f910 con 0x7f57980ce730 2023-10-19T10:38:00.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579fbb2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579806fbf0 con 0x7f5798145880 2023-10-19T10:38:00.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579d94e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f57980b26e0 0x7f57980ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46790/0 (socket says 172.21.15.50:46790) 2023-10-19T10:38:00.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579d94e640 1 -- 172.21.15.50:0/2417479786 learned_addr learned my addr 172.21.15.50:0/2417479786 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:00.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.080+0000 7f579d14d640 1 --2- 172.21.15.50:0/2417479786 >> v2:172.21.15.50:3300/0 conn(0x7f57980ce730 0x7f579806f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579e14f640 1 --2- 172.21.15.50:0/2417479786 >> v2:172.21.15.183:3300/0 conn(0x7f5798145880 0x7f579806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579d14d640 1 -- 172.21.15.50:0/2417479786 >> v2:172.21.15.183:3300/0 conn(0x7f5798145880 msgr2=0x7f579806aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:00.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579d14d640 1 --2- 172.21.15.50:0/2417479786 >> v2:172.21.15.183:3300/0 conn(0x7f5798145880 0x7f579806aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579d14d640 1 -- 172.21.15.50:0/2417479786 >> v1:172.21.15.50:6789/0 conn(0x7f57980b26e0 legacy=0x7f57980ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579d14d640 1 -- 172.21.15.50:0/2417479786 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579806fed0 con 0x7f57980ce730 2023-10-19T10:38:00.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579d14d640 1 --2- 172.21.15.50:0/2417479786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f579806f2a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5798071850 tx=0x7f578c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60eca7879e246f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.081+0000 7f579c94c640 1 -- 172.21.15.50:0/2417479786 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c02e070 con 0x7f57980ce730 2023-10-19T10:38:00.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579c94c640 1 -- 172.21.15.50:0/2417479786 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c0315a0 con 0x7f57980ce730 2023-10-19T10:38:00.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579c94c640 1 -- 172.21.15.50:0/2417479786 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c031930 con 0x7f57980ce730 2023-10-19T10:38:00.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 -- 172.21.15.50:0/2417479786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 msgr2=0x7f579806f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 --2- 172.21.15.50:0/2417479786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f579806f2a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5798071850 tx=0x7f578c02b840 comp rx=0 tx=0).stop 2023-10-19T10:38:00.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 -- 172.21.15.50:0/2417479786 shutdown_connections 2023-10-19T10:38:00.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 --2- 172.21.15.50:0/2417479786 >> v2:172.21.15.183:3300/0 conn(0x7f5798145880 0x7f579806aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 --2- 172.21.15.50:0/2417479786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f579806f2a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 -- 172.21.15.50:0/2417479786 >> 172.21.15.50:0/2417479786 conn(0x7f57980ccf60 msgr2=0x7f57980cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:00.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 -- 172.21.15.50:0/2417479786 shutdown_connections 2023-10-19T10:38:00.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.082+0000 7f579fbb2640 1 -- 172.21.15.50:0/2417479786 wait complete. 2023-10-19T10:38:00.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.083+0000 7f579fbb2640 1 Processor -- start 2023-10-19T10:38:00.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.083+0000 7f579fbb2640 1 -- start start 2023-10-19T10:38:00.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579fbb2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57980b26e0 0x7f57980b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579fbb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f5798160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579fbb2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5798145880 0x7f579815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579fbb2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980adcc0 con 0x7f57980ce730 2023-10-19T10:38:00.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579fbb2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b43b0 con 0x7f5798145880 2023-10-19T10:38:00.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579fbb2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b3df0 con 0x7f57980b26e0 2023-10-19T10:38:00.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579d14d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f5798160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579d14d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f5798160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55452/0 (socket says 172.21.15.50:55452) 2023-10-19T10:38:00.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579d14d640 1 -- 172.21.15.50:0/486496176 learned_addr learned my addr 172.21.15.50:0/486496176 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:00.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579e14f640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5798145880 0x7f579815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.084+0000 7f579d94e640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57980b26e0 0x7f57980b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d14d640 1 -- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57980b26e0 msgr2=0x7f57980b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d14d640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57980b26e0 0x7f57980b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d14d640 1 -- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5798145880 msgr2=0x7f579815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d14d640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5798145880 0x7f579815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d14d640 1 -- 172.21.15.50:0/486496176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009400 con 0x7f57980ce730 2023-10-19T10:38:00.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d94e640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57980b26e0 0x7f57980b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:00.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579d14d640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f5798160180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f578c000c00 tx=0x7f578c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c031f10 con 0x7f57980ce730 2023-10-19T10:38:00.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c032840 con 0x7f57980ce730 2023-10-19T10:38:00.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57980b40d0 con 0x7f57980ce730 2023-10-19T10:38:00.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c032bd0 con 0x7f57980ce730 2023-10-19T10:38:00.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.085+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57980ae1d0 con 0x7f57980ce730 2023-10-19T10:38:00.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.086+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57980ae3c0 con 0x7f57980ce730 2023-10-19T10:38:00.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.086+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f578c032d30 con 0x7f57980ce730 2023-10-19T10:38:00.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.087+0000 7f57967fc640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f578406a050 0x7f578406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.087+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f578c0aa650 con 0x7f57980ce730 2023-10-19T10:38:00.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.087+0000 7f579d94e640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f578406a050 0x7f578406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.087+0000 7f579d94e640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f578406a050 0x7f578406c510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57980ab990 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.091+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f578c075ac0 con 0x7f57980ce730 2023-10-19T10:38:00.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.264+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f579804e420 con 0x7f57980ce730 2023-10-19T10:38:00.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.264+0000 7f57967fc640 1 -- 172.21.15.50:0/486496176 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f578c0790e0 con 0x7f57980ce730 2023-10-19T10:38:00.267 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:00.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f578406a050 msgr2=0x7f578406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f578406a050 0x7f578406c510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57980ab990 tx=0x7f5788009040 comp rx=0 tx=0).stop 2023-10-19T10:38:00.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 msgr2=0x7f5798160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f5798160180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f578c000c00 tx=0x7f578c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 shutdown_connections 2023-10-19T10:38:00.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f578406a050 0x7f578406c510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5798145880 0x7f579815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57980ce730 0x7f5798160180 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 --2- 172.21.15.50:0/486496176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57980b26e0 0x7f57980b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 >> 172.21.15.50:0/486496176 conn(0x7f57980ccf60 msgr2=0x7f57980ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:00.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 shutdown_connections 2023-10-19T10:38:00.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.266+0000 7f579fbb2640 1 -- 172.21.15.50:0/486496176 wait complete. 2023-10-19T10:38:00.279 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:38:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.384+0000 7f1fade5f640 1 Processor -- start 2023-10-19T10:38:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.384+0000 7f1fade5f640 1 -- start start 2023-10-19T10:38:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fade5f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1fa815a370 0x7f1fa806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fade5f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fa80b1f30 0x7f1fa80b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fade5f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fade5f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa80b2900 con 0x7f1fa80b1f30 2023-10-19T10:38:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fade5f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa80caf80 con 0x7f1fa815a370 2023-10-19T10:38:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fade5f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80d02e0 con 0x7f1fa815de70 2023-10-19T10:38:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fa80b1f30 0x7f1fa80b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fa80b1f30 0x7f1fa80b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55458/0 (socket says 172.21.15.50:55458) 2023-10-19T10:38:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/1013698838 learned_addr learned my addr 172.21.15.50:0/1013698838 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa77fe640 1 --2- 172.21.15.50:0/1013698838 >> v2:172.21.15.183:3300/0 conn(0x7f1fa815a370 0x7f1fa806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa7fff640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/1013698838 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 msgr2=0x7f1fa815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:00.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.385+0000 7f1fa6ffd640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.386+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/1013698838 >> v2:172.21.15.183:3300/0 conn(0x7f1fa815a370 msgr2=0x7f1fa806fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:00.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.386+0000 7f1fa6ffd640 1 --2- 172.21.15.50:0/1013698838 >> v2:172.21.15.183:3300/0 conn(0x7f1fa815a370 0x7f1fa806fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.386+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/1013698838 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa815a750 con 0x7f1fa80b1f30 2023-10-19T10:38:00.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.386+0000 7f1fa7fff640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.386+0000 7f1fa6ffd640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa80b1f30 0x7f1fa80b23c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8072040 tx=0x7f1f9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4c0e17ed6876 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.386+0000 7f1fa67fc640 1 -- 172.21.15.50:0/1013698838 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c031870 con 0x7f1fa80b1f30 2023-10-19T10:38:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fa67fc640 1 -- 172.21.15.50:0/1013698838 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0319d0 con 0x7f1fa80b1f30 2023-10-19T10:38:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fa67fc640 1 -- 172.21.15.50:0/1013698838 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c031d60 con 0x7f1fa80b1f30 2023-10-19T10:38:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 -- 172.21.15.50:0/1013698838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa80b1f30 msgr2=0x7f1fa80b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa80b1f30 0x7f1fa80b23c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8072040 tx=0x7f1f9c02b910 comp rx=0 tx=0).stop 2023-10-19T10:38:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 -- 172.21.15.50:0/1013698838 shutdown_connections 2023-10-19T10:38:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 --2- 172.21.15.50:0/1013698838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa80b1f30 0x7f1fa80b23c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 --2- 172.21.15.50:0/1013698838 >> v2:172.21.15.183:3300/0 conn(0x7f1fa815a370 0x7f1fa806fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 -- 172.21.15.50:0/1013698838 >> 172.21.15.50:0/1013698838 conn(0x7f1fa80ccf30 msgr2=0x7f1fa80cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:00.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 -- 172.21.15.50:0/1013698838 shutdown_connections 2023-10-19T10:38:00.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.387+0000 7f1fade5f640 1 -- 172.21.15.50:0/1013698838 wait complete. 2023-10-19T10:38:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.388+0000 7f1fade5f640 1 Processor -- start 2023-10-19T10:38:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.388+0000 7f1fade5f640 1 -- start start 2023-10-19T10:38:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fade5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fa80b1f30 0x7f1fa8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fade5f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 0x7f1fa815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fade5f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fade5f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80ae840 con 0x7f1fa815a370 2023-10-19T10:38:00.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fade5f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815bf50 con 0x7f1fa80b1f30 2023-10-19T10:38:00.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fade5f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815c230 con 0x7f1fa815de70 2023-10-19T10:38:00.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fa6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 0x7f1fa815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fa6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 0x7f1fa815b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55466/0 (socket says 172.21.15.50:55466) 2023-10-19T10:38:00.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/3779174809 learned_addr learned my addr 172.21.15.50:0/3779174809 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:00.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fa7fff640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.389+0000 7f1fa77fe640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fa80b1f30 0x7f1fa8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 msgr2=0x7f1fa815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fa6ffd640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fa80b1f30 msgr2=0x7f1fa8159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fa6ffd640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fa80b1f30 0x7f1fa8159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fa6ffd640 1 -- 172.21.15.50:0/3779174809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0094d0 con 0x7f1fa815a370 2023-10-19T10:38:00.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fa6ffd640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 0x7f1fa815b5e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c02b950 tx=0x7f1f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c03abe0 con 0x7f1fa815a370 2023-10-19T10:38:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c03ad60 con 0x7f1fa815a370 2023-10-19T10:38:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa815c510 con 0x7f1fa815a370 2023-10-19T10:38:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c044630 con 0x7f1fa815a370 2023-10-19T10:38:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.390+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa815cad0 con 0x7f1fa815a370 2023-10-19T10:38:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.391+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa815ccc0 con 0x7f1fa815a370 2023-10-19T10:38:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.391+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1f9c04d450 con 0x7f1fa815a370 2023-10-19T10:38:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.392+0000 7f1f8ffff640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f9406a050 0x7f1f9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.392+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1f9c02d040 con 0x7f1fa815a370 2023-10-19T10:38:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.392+0000 7f1fa77fe640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f9406a050 0x7f1f9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.393+0000 7f1fa77fe640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f9406a050 0x7f1f9406c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009ac0 tx=0x7f1f98009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.396+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1f9c076150 con 0x7f1fa815a370 2023-10-19T10:38:00.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.588+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1fa8073990 con 0x7f1fa815a370 2023-10-19T10:38:00.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.589+0000 7f1f8ffff640 1 -- 172.21.15.50:0/3779174809 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1f9c035020 con 0x7f1fa815a370 2023-10-19T10:38:00.592 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:38:00.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f9406a050 msgr2=0x7f1f9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f9406a050 0x7f1f9406c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009ac0 tx=0x7f1f98009190 comp rx=0 tx=0).stop 2023-10-19T10:38:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 msgr2=0x7f1fa815b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 0x7f1fa815b5e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c02b950 tx=0x7f1f9c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 shutdown_connections 2023-10-19T10:38:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1f9406a050 0x7f1f9406c510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fa815de70 0x7f1fa815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fa815a370 0x7f1fa815b5e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 --2- 172.21.15.50:0/3779174809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fa80b1f30 0x7f1fa8159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 >> 172.21.15.50:0/3779174809 conn(0x7f1fa80ccf30 msgr2=0x7f1fa80b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.590+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 shutdown_connections 2023-10-19T10:38:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.591+0000 7f1fade5f640 1 -- 172.21.15.50:0/3779174809 wait complete. 2023-10-19T10:38:00.604 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:38:00.604 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:00.624 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:00.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.728+0000 7f43f1cdc640 1 Processor -- start 2023-10-19T10:38:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.728+0000 7f43f1cdc640 1 -- start start 2023-10-19T10:38:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43f1cdc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f43ec15c570 0x7f43ec0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43f1cdc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec06aa30 con 0x7f43ec15c950 2023-10-19T10:38:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43f1cdc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec06b2d0 con 0x7f43ec0cd920 2023-10-19T10:38:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43f1cdc640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0ce090 con 0x7f43ec15c570 2023-10-19T10:38:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f43ec15c570 0x7f43ec0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eaffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f43ec15c950 0x7f43ec06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46800/0 (socket says 172.21.15.50:46800) 2023-10-19T10:38:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f43ec15c570 0x7f43ec0d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55472/0 (socket says 172.21.15.50:55472) 2023-10-19T10:38:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eaffd640 1 -- 172.21.15.50:0/1865445201 learned_addr learned my addr 172.21.15.50:0/1865445201 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eb7fe640 1 -- 172.21.15.50:0/1865445201 >> v1:172.21.15.183:6789/0 conn(0x7f43ec0cd920 legacy=0x7f43ec0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eb7fe640 1 -- 172.21.15.50:0/1865445201 >> v1:172.21.15.50:6789/0 conn(0x7f43ec15c950 legacy=0x7f43ec06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.729+0000 7f43eb7fe640 1 -- 172.21.15.50:0/1865445201 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec06ac80 con 0x7f43ec15c570 2023-10-19T10:38:00.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.730+0000 7f43eb7fe640 1 --2- 172.21.15.50:0/1865445201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c570 0x7f43ec0d0e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009790 tx=0x7f43dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f73221441b91bcf server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.730+0000 7f43ea7fc640 1 -- 172.21.15.50:0/1865445201 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc031830 con 0x7f43ec15c570 2023-10-19T10:38:00.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.730+0000 7f43ea7fc640 1 -- 172.21.15.50:0/1865445201 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc031990 con 0x7f43ec15c570 2023-10-19T10:38:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.730+0000 7f43ea7fc640 1 -- 172.21.15.50:0/1865445201 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc031d20 con 0x7f43ec15c570 2023-10-19T10:38:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 -- 172.21.15.50:0/1865445201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c570 msgr2=0x7f43ec0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/1865445201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c570 0x7f43ec0d0e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009790 tx=0x7f43dc02b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 -- 172.21.15.50:0/1865445201 shutdown_connections 2023-10-19T10:38:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/1865445201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c570 0x7f43ec0d0e30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 -- 172.21.15.50:0/1865445201 >> 172.21.15.50:0/1865445201 conn(0x7f43ec089ff0 msgr2=0x7f43ec09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 -- 172.21.15.50:0/1865445201 shutdown_connections 2023-10-19T10:38:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.731+0000 7f43f1cdc640 1 -- 172.21.15.50:0/1865445201 wait complete. 2023-10-19T10:38:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.732+0000 7f43f1cdc640 1 Processor -- start 2023-10-19T10:38:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.732+0000 7f43f1cdc640 1 -- start start 2023-10-19T10:38:00.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43f1cdc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec0cd920 0x7f43ec14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43f1cdc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43ec15c570 0x7f43ec0af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43f1cdc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 0x7f43ec0af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43f1cdc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec14caf0 con 0x7f43ec15c950 2023-10-19T10:38:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43f1cdc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1efb10 con 0x7f43ec0cd920 2023-10-19T10:38:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43f1cdc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec14be40 con 0x7f43ec15c570 2023-10-19T10:38:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 0x7f43ec0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 0x7f43ec0af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55474/0 (socket says 172.21.15.50:55474) 2023-10-19T10:38:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 -- 172.21.15.50:0/4020019724 learned_addr learned my addr 172.21.15.50:0/4020019724 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43eb7fe640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec0cd920 0x7f43ec14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43eaffd640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43ec15c570 0x7f43ec0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 -- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43ec15c570 msgr2=0x7f43ec0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43ec15c570 0x7f43ec0af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 -- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec0cd920 msgr2=0x7f43ec14b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec0cd920 0x7f43ec14b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 -- 172.21.15.50:0/4020019724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0093a0 con 0x7f43ec15c950 2023-10-19T10:38:00.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.733+0000 7f43ebfff640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 0x7f43ec0af880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43e400c790 tx=0x7f43e400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.734+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4003b70 con 0x7f43ec15c950 2023-10-19T10:38:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.734+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e4003cf0 con 0x7f43ec15c950 2023-10-19T10:38:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.734+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4005310 con 0x7f43ec15c950 2023-10-19T10:38:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.734+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec1f3020 con 0x7f43ec15c950 2023-10-19T10:38:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.734+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec14c4a0 con 0x7f43ec15c950 2023-10-19T10:38:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.735+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec14cca0 con 0x7f43ec15c950 2023-10-19T10:38:00.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.735+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f43e4005470 con 0x7f43ec15c950 2023-10-19T10:38:00.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.735+0000 7f43d3fff640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f43d806a050 0x7f43d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:00.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.736+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f43e4088700 con 0x7f43ec15c950 2023-10-19T10:38:00.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.736+0000 7f43eb7fe640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f43d806a050 0x7f43d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:00.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.736+0000 7f43eb7fe640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f43d806a050 0x7f43d806c510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f43dc036000 tx=0x7f43dc03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:00.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.740+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f43ec14cca0 con 0x7f43ec15c950 2023-10-19T10:38:00.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.915+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43ec14cca0 con 0x7f43ec15c950 2023-10-19T10:38:00.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.916+0000 7f43d3fff640 1 -- 172.21.15.50:0/4020019724 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f43ec14cca0 con 0x7f43ec15c950 2023-10-19T10:38:00.919 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:00.919 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:00.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f43d806a050 msgr2=0x7f43d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f43d806a050 0x7f43d806c510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f43dc036000 tx=0x7f43dc03c040 comp rx=0 tx=0).stop 2023-10-19T10:38:00.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 msgr2=0x7f43ec0af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:00.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 0x7f43ec0af880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43e400c790 tx=0x7f43e400cc50 comp rx=0 tx=0).stop 2023-10-19T10:38:00.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 shutdown_connections 2023-10-19T10:38:00.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f43d806a050 0x7f43d806c510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec15c950 0x7f43ec0af880 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43ec15c570 0x7f43ec0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 --2- 172.21.15.50:0/4020019724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec0cd920 0x7f43ec14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:00.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 >> 172.21.15.50:0/4020019724 conn(0x7f43ec089ff0 msgr2=0x7f43ec072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:00.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 shutdown_connections 2023-10-19T10:38:00.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:00.918+0000 7f43f1cdc640 1 -- 172.21.15.50:0/4020019724 wait complete. 2023-10-19T10:38:00.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:38:00.932 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.039+0000 7f1822051640 1 Processor -- start 2023-10-19T10:38:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.039+0000 7f1822051640 1 -- start start 2023-10-19T10:38:01.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.040+0000 7f1822051640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f181c151220 0x7f181c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.040+0000 7f1822051640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181c06c850 con 0x7f181c15e680 2023-10-19T10:38:01.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.040+0000 7f1822051640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181c06c290 con 0x7f181c06bdf0 2023-10-19T10:38:01.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.040+0000 7f1822051640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181c06c570 con 0x7f181c151220 2023-10-19T10:38:01.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.040+0000 7f181bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f181c15e680 0x7f181c0ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46802/0 (socket says 172.21.15.50:46802) 2023-10-19T10:38:01.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.040+0000 7f181bfff640 1 -- 172.21.15.50:0/977431477 learned_addr learned my addr 172.21.15.50:0/977431477 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:01.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181b7fe640 1 --2- 172.21.15.50:0/977431477 >> v2:172.21.15.183:3300/0 conn(0x7f181c151220 0x7f181c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237208792 0 0) 0x7f181c06c850 con 0x7f181c15e680 2023-10-19T10:38:01.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1804003520 con 0x7f181c15e680 2023-10-19T10:38:01.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772736799 0 0) 0x7f181c06c290 con 0x7f181c06bdf0 2023-10-19T10:38:01.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f181c06c850 con 0x7f181c06bdf0 2023-10-19T10:38:01.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253711133 0 0) 0x7f1804003520 con 0x7f181c15e680 2023-10-19T10:38:01.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 >> v2:172.21.15.183:3300/0 conn(0x7f181c151220 msgr2=0x7f181c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:01.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 --2- 172.21.15.50:0/977431477 >> v2:172.21.15.183:3300/0 conn(0x7f181c151220 0x7f181c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 >> v1:172.21.15.183:6789/0 conn(0x7f181c06bdf0 legacy=0x7f181c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0ca050 con 0x7f181c15e680 2023-10-19T10:38:01.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f18140019f0 con 0x7f181c15e680 2023-10-19T10:38:01.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 >> v1:172.21.15.50:6789/0 conn(0x7f181c15e680 legacy=0x7f181c0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1804003b00 0x7f1804003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f1804004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18040051f0 0x7f180400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c06c570 con 0x7f18040047c0 2023-10-19T10:38:01.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180400c150 con 0x7f1804003b00 2023-10-19T10:38:01.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180400c310 con 0x7f18040051f0 2023-10-19T10:38:01.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181b7fe640 1 --2- 172.21.15.50:0/977431477 >> v2:172.21.15.183:3300/0 conn(0x7f181c151220 0x7f181c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:01.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181affd640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f1804004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181bfff640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18040051f0 0x7f180400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181b7fe640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1804003b00 0x7f1804003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181affd640 1 -- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18040051f0 msgr2=0x7f180400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:01.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181affd640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18040051f0 0x7f180400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181affd640 1 -- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1804003b00 msgr2=0x7f1804003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:01.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181affd640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1804003b00 0x7f1804003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181affd640 1 -- 172.21.15.50:0/977431477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804003520 con 0x7f18040047c0 2023-10-19T10:38:01.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181b7fe640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1804003b00 0x7f1804003f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:01.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.041+0000 7f181bfff640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18040051f0 0x7f180400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:01.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.042+0000 7f181bfff640 1 -- 172.21.15.50:0/977431477 reap_dead start 2023-10-19T10:38:01.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.042+0000 7f181affd640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f1804004c40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1810001890 tx=0x7f18040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a558c7e6b9f772ec server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.042+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100052d0 con 0x7f18040047c0 2023-10-19T10:38:01.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.042+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1810005450 con 0x7f18040047c0 2023-10-19T10:38:01.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.042+0000 7f181a7fc640 1 -- 172.21.15.50:0/977431477 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810003830 con 0x7f18040047c0 2023-10-19T10:38:01.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 -- 172.21.15.50:0/977431477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 msgr2=0x7f1804004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f1804004c40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1810001890 tx=0x7f18040057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 -- 172.21.15.50:0/977431477 shutdown_connections 2023-10-19T10:38:01.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 --2- 172.21.15.50:0/977431477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f1804004c40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 -- 172.21.15.50:0/977431477 >> 172.21.15.50:0/977431477 conn(0x7f181c0afbe0 msgr2=0x7f181c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 -- 172.21.15.50:0/977431477 shutdown_connections 2023-10-19T10:38:01.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.043+0000 7f1822051640 1 -- 172.21.15.50:0/977431477 wait complete. 2023-10-19T10:38:01.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.044+0000 7f1822051640 1 Processor -- start 2023-10-19T10:38:01.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.044+0000 7f1822051640 1 -- start start 2023-10-19T10:38:01.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f1822051640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f181c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f1822051640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c0c3e40 0x7f181c143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f1822051640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f181c1440b0 0x7f181c0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f1822051640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c0c8ad0 con 0x7f18040047c0 2023-10-19T10:38:01.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f1822051640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c1445f0 con 0x7f181c0c3e40 2023-10-19T10:38:01.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f1822051640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181c1448d0 con 0x7f181c1440b0 2023-10-19T10:38:01.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f181c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f181c0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55492/0 (socket says 172.21.15.50:55492) 2023-10-19T10:38:01.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 -- 172.21.15.50:0/696054425 learned_addr learned my addr 172.21.15.50:0/696054425 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:01.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 -- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f181c1440b0 msgr2=0x7f181c0ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f181c1440b0 0x7f181c0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 -- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c0c3e40 msgr2=0x7f181c143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c0c3e40 0x7f181c143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.045+0000 7f181b7fe640 1 -- 172.21.15.50:0/696054425 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0c8760 con 0x7f18040047c0 2023-10-19T10:38:01.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.046+0000 7f181b7fe640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f181c0cc090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f180c011860 tx=0x7f180c011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.046+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00fb50 con 0x7f18040047c0 2023-10-19T10:38:01.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.046+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c00fcb0 con 0x7f18040047c0 2023-10-19T10:38:01.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.046+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c0c8410 con 0x7f18040047c0 2023-10-19T10:38:01.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.046+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c015460 con 0x7f18040047c0 2023-10-19T10:38:01.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.046+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c0ad2e0 con 0x7f18040047c0 2023-10-19T10:38:01.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.047+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c0c85a0 con 0x7f18040047c0 2023-10-19T10:38:01.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.047+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f180c0155c0 con 0x7f18040047c0 2023-10-19T10:38:01.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.047+0000 7f1803fff640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f180806a050 0x7f180806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.048+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f180c089400 con 0x7f18040047c0 2023-10-19T10:38:01.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.048+0000 7f181affd640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f180806a050 0x7f180806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.048+0000 7f181affd640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f180806a050 0x7f180806c510 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1810034000 tx=0x7f1810030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.052+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f180c0547c0 con 0x7f18040047c0 2023-10-19T10:38:01.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.225+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f181c144a90 con 0x7f18040047c0 2023-10-19T10:38:01.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.226+0000 7f1803fff640 1 -- 172.21.15.50:0/696054425 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f180c057de0 con 0x7f18040047c0 2023-10-19T10:38:01.229 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:01.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f180806a050 msgr2=0x7f180806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f180806a050 0x7f180806c510 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1810034000 tx=0x7f1810030000 comp rx=0 tx=0).stop 2023-10-19T10:38:01.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 msgr2=0x7f181c0cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f181c0cc090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f180c011860 tx=0x7f180c011d20 comp rx=0 tx=0).stop 2023-10-19T10:38:01.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 shutdown_connections 2023-10-19T10:38:01.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f180806a050 0x7f180806c510 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f181c1440b0 0x7f181c0ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c0c3e40 0x7f181c143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 --2- 172.21.15.50:0/696054425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18040047c0 0x7f181c0cc090 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 >> 172.21.15.50:0/696054425 conn(0x7f181c0afbe0 msgr2=0x7f181c15c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 shutdown_connections 2023-10-19T10:38:01.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.228+0000 7f1822051640 1 -- 172.21.15.50:0/696054425 wait complete. 2023-10-19T10:38:01.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.347+0000 7f7840c21640 1 Processor -- start 2023-10-19T10:38:01.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.347+0000 7f7840c21640 1 -- start start 2023-10-19T10:38:01.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f7840c21640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f783c0b3090 0x7f783c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f7840c21640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0af250 con 0x7f783c161210 2023-10-19T10:38:01.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f7840c21640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0af510 con 0x7f783c0b38f0 2023-10-19T10:38:01.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f7840c21640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c0b0190 con 0x7f783c0b3090 2023-10-19T10:38:01.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f783ad9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f783c161210 0x7f783c0b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46808/0 (socket says 172.21.15.50:46808) 2023-10-19T10:38:01.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f783ad9d640 1 -- 172.21.15.50:0/2184177149 learned_addr learned my addr 172.21.15.50:0/2184177149 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:01.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.348+0000 7f7839d9b640 1 --2- 172.21.15.50:0/2184177149 >> v2:172.21.15.50:3300/0 conn(0x7f783c0b3090 0x7f783c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.349+0000 7f7839d9b640 1 -- 172.21.15.50:0/2184177149 >> v1:172.21.15.183:6789/0 conn(0x7f783c0b38f0 legacy=0x7f783c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.349+0000 7f7839d9b640 1 -- 172.21.15.50:0/2184177149 >> v1:172.21.15.50:6789/0 conn(0x7f783c161210 legacy=0x7f783c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.349+0000 7f7839d9b640 1 -- 172.21.15.50:0/2184177149 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c0af7f0 con 0x7f783c0b3090 2023-10-19T10:38:01.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.349+0000 7f7839d9b640 1 --2- 172.21.15.50:0/2184177149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c0b3090 0x7f783c0d30e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f78240097a0 tx=0x7f782402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575d90f6a23b6af7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f783959a640 1 -- 172.21.15.50:0/2184177149 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824030610 con 0x7f783c0b3090 2023-10-19T10:38:01.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f783959a640 1 -- 172.21.15.50:0/2184177149 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824030770 con 0x7f783c0b3090 2023-10-19T10:38:01.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f783959a640 1 -- 172.21.15.50:0/2184177149 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824030b00 con 0x7f783c0b3090 2023-10-19T10:38:01.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 -- 172.21.15.50:0/2184177149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c0b3090 msgr2=0x7f783c0d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 --2- 172.21.15.50:0/2184177149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c0b3090 0x7f783c0d30e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f78240097a0 tx=0x7f782402b960 comp rx=0 tx=0).stop 2023-10-19T10:38:01.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 -- 172.21.15.50:0/2184177149 shutdown_connections 2023-10-19T10:38:01.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 --2- 172.21.15.50:0/2184177149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c0b3090 0x7f783c0d30e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 -- 172.21.15.50:0/2184177149 >> 172.21.15.50:0/2184177149 conn(0x7f783c09b960 msgr2=0x7f783c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 -- 172.21.15.50:0/2184177149 shutdown_connections 2023-10-19T10:38:01.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.350+0000 7f7840c21640 1 -- 172.21.15.50:0/2184177149 wait complete. 2023-10-19T10:38:01.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.351+0000 7f7840c21640 1 Processor -- start 2023-10-19T10:38:01.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 -- start start 2023-10-19T10:38:01.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f783c0b3090 0x7f783c0b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783c0b38f0 0x7f783c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 0x7f783c15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0cabb0 con 0x7f783c161210 2023-10-19T10:38:01.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c15b430 con 0x7f783c0b38f0 2023-10-19T10:38:01.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f7840c21640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c15f2e0 con 0x7f783c0b3090 2023-10-19T10:38:01.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f783ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 0x7f783c15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f783ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 0x7f783c15aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55522/0 (socket says 172.21.15.50:55522) 2023-10-19T10:38:01.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.352+0000 7f783ad9d640 1 -- 172.21.15.50:0/200180258 learned_addr learned my addr 172.21.15.50:0/200180258 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:01.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7839d9b640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783c0b38f0 0x7f783c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783a59c640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f783c0b3090 0x7f783c0b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783ad9d640 1 -- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f783c0b3090 msgr2=0x7f783c0b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783ad9d640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f783c0b3090 0x7f783c0b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783ad9d640 1 -- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783c0b38f0 msgr2=0x7f783c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783ad9d640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783c0b38f0 0x7f783c0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783ad9d640 1 -- 172.21.15.50:0/200180258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824009400 con 0x7f783c161210 2023-10-19T10:38:01.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7839d9b640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783c0b38f0 0x7f783c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:01.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783a59c640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f783c0b3090 0x7f783c0b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:01.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f783ad9d640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 0x7f783c15aef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f782c00caa0 tx=0x7f782c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00ac00 con 0x7f783c161210 2023-10-19T10:38:01.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c00ad60 con 0x7f783c161210 2023-10-19T10:38:01.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c0cd660 con 0x7f783c161210 2023-10-19T10:38:01.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c012600 con 0x7f783c161210 2023-10-19T10:38:01.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.353+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c0cdc20 con 0x7f783c161210 2023-10-19T10:38:01.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.354+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c0cacf0 con 0x7f783c161210 2023-10-19T10:38:01.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.355+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f782c007070 con 0x7f783c161210 2023-10-19T10:38:01.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.355+0000 7f7832ffd640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f781c06a050 0x7f781c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.355+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f782c088630 con 0x7f783c161210 2023-10-19T10:38:01.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.355+0000 7f783a59c640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f781c06a050 0x7f781c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.356+0000 7f783a59c640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f781c06a050 0x7f781c06c510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f783c0c6030 tx=0x7f7834009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.359+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f782c053b30 con 0x7f783c161210 2023-10-19T10:38:01.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.536+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f783c0cde10 con 0x7f783c161210 2023-10-19T10:38:01.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.537+0000 7f7832ffd640 1 -- 172.21.15.50:0/200180258 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f782c057150 con 0x7f783c161210 2023-10-19T10:38:01.540 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f781c06a050 msgr2=0x7f781c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f781c06a050 0x7f781c06c510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f783c0c6030 tx=0x7f7834009040 comp rx=0 tx=0).stop 2023-10-19T10:38:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 msgr2=0x7f783c15aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 0x7f783c15aef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f782c00caa0 tx=0x7f782c00cf60 comp rx=0 tx=0).stop 2023-10-19T10:38:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 shutdown_connections 2023-10-19T10:38:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f781c06a050 0x7f781c06c510 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f783c161210 0x7f783c15aef0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783c0b38f0 0x7f783c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 --2- 172.21.15.50:0/200180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f783c0b3090 0x7f783c0b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 >> 172.21.15.50:0/200180258 conn(0x7f783c09b960 msgr2=0x7f783c0b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 shutdown_connections 2023-10-19T10:38:01.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.539+0000 7f7840c21640 1 -- 172.21.15.50:0/200180258 wait complete. 2023-10-19T10:38:01.553 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:38:01.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.660+0000 7f03de185640 1 Processor -- start 2023-10-19T10:38:01.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.660+0000 7f03de185640 1 -- start start 2023-10-19T10:38:01.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03de185640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f03d80b4070 0x7f03d80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03de185640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d815c960 0x7f03d80d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03de185640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d80b4990 con 0x7f03d815cdf0 2023-10-19T10:38:01.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03de185640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d815d280 con 0x7f03d80b4070 2023-10-19T10:38:01.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03de185640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d815d560 con 0x7f03d815c960 2023-10-19T10:38:01.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f03d80b4070 0x7f03d80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d7fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f03d815cdf0 0x7f03d8160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46824/0 (socket says 172.21.15.50:46824) 2023-10-19T10:38:01.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f03d80b4070 0x7f03d80b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55536/0 (socket says 172.21.15.50:55536) 2023-10-19T10:38:01.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d7fff640 1 -- 172.21.15.50:0/3897632443 learned_addr learned my addr 172.21.15.50:0/3897632443 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:01.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d6ffd640 1 --2- 172.21.15.50:0/3897632443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d815c960 0x7f03d80d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d77fe640 1 -- 172.21.15.50:0/3897632443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d815c960 msgr2=0x7f03d80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:01.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d77fe640 1 --2- 172.21.15.50:0/3897632443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d815c960 0x7f03d80d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d77fe640 1 -- 172.21.15.50:0/3897632443 >> v1:172.21.15.50:6789/0 conn(0x7f03d815cdf0 legacy=0x7f03d8160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.661+0000 7f03d77fe640 1 -- 172.21.15.50:0/3897632443 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d815d840 con 0x7f03d80b4070 2023-10-19T10:38:01.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.662+0000 7f03d77fe640 1 --2- 172.21.15.50:0/3897632443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d80b4070 0x7f03d80b4450 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009790 tx=0x7f03c802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e638df89a6d83be server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.662+0000 7f03d67fc640 1 -- 172.21.15.50:0/3897632443 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8031800 con 0x7f03d80b4070 2023-10-19T10:38:01.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.662+0000 7f03d67fc640 1 -- 172.21.15.50:0/3897632443 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8031960 con 0x7f03d80b4070 2023-10-19T10:38:01.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.662+0000 7f03d67fc640 1 -- 172.21.15.50:0/3897632443 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8031cf0 con 0x7f03d80b4070 2023-10-19T10:38:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 -- 172.21.15.50:0/3897632443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d80b4070 msgr2=0x7f03d80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 --2- 172.21.15.50:0/3897632443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d80b4070 0x7f03d80b4450 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009790 tx=0x7f03c802b8a0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 -- 172.21.15.50:0/3897632443 shutdown_connections 2023-10-19T10:38:01.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 --2- 172.21.15.50:0/3897632443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d815c960 0x7f03d80d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 --2- 172.21.15.50:0/3897632443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d80b4070 0x7f03d80b4450 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 -- 172.21.15.50:0/3897632443 >> 172.21.15.50:0/3897632443 conn(0x7f03d809b970 msgr2=0x7f03d80d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 -- 172.21.15.50:0/3897632443 shutdown_connections 2023-10-19T10:38:01.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.663+0000 7f03de185640 1 -- 172.21.15.50:0/3897632443 wait complete. 2023-10-19T10:38:01.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.664+0000 7f03de185640 1 Processor -- start 2023-10-19T10:38:01.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.664+0000 7f03de185640 1 -- start start 2023-10-19T10:38:01.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03de185640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d80b4070 0x7f03d8150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03de185640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 0x7f03d80c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03de185640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03d80b7f10 0x7f03d80b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03de185640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d80c69e0 con 0x7f03d815c960 2023-10-19T10:38:01.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03de185640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d80c6cc0 con 0x7f03d80b7f10 2023-10-19T10:38:01.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03de185640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d80c6fa0 con 0x7f03d80b4070 2023-10-19T10:38:01.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03d6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 0x7f03d80c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03d6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 0x7f03d80c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55552/0 (socket says 172.21.15.50:55552) 2023-10-19T10:38:01.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03d6ffd640 1 -- 172.21.15.50:0/3487542477 learned_addr learned my addr 172.21.15.50:0/3487542477 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:01.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03d7fff640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03d80b7f10 0x7f03d80b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.665+0000 7f03d6ffd640 1 -- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d80b4070 msgr2=0x7f03d8150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d77fe640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d80b4070 0x7f03d8150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d6ffd640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d80b4070 0x7f03d8150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d6ffd640 1 -- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03d80b7f10 msgr2=0x7f03d80b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d6ffd640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03d80b7f10 0x7f03d80b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d6ffd640 1 -- 172.21.15.50:0/3487542477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c80093a0 con 0x7f03d815c960 2023-10-19T10:38:01.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d7fff640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03d80b7f10 0x7f03d80b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:01.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d77fe640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d80b4070 0x7f03d8150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:01.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03d6ffd640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 0x7f03d80c6090 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00d800 tx=0x7f03cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00b6c0 con 0x7f03d815c960 2023-10-19T10:38:01.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00b820 con 0x7f03d815c960 2023-10-19T10:38:01.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d80c7280 con 0x7f03d815c960 2023-10-19T10:38:01.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00bbb0 con 0x7f03d815c960 2023-10-19T10:38:01.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.666+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d8151130 con 0x7f03d815c960 2023-10-19T10:38:01.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.667+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d804e420 con 0x7f03d815c960 2023-10-19T10:38:01.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.667+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f03cc00bd10 con 0x7f03d815c960 2023-10-19T10:38:01.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.668+0000 7f03bffff640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f03c406a050 0x7f03c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.668+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f03cc088df0 con 0x7f03d815c960 2023-10-19T10:38:01.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.668+0000 7f03d77fe640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f03c406a050 0x7f03c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.669+0000 7f03d77fe640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f03c406a050 0x7f03c406c510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03c8036000 tx=0x7f03c8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.672+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f03cc0541b0 con 0x7f03d815c960 2023-10-19T10:38:01.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.845+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03d808c720 con 0x7f03d815c960 2023-10-19T10:38:01.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.846+0000 7f03bffff640 1 -- 172.21.15.50:0/3487542477 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03cc0577d0 con 0x7f03d815c960 2023-10-19T10:38:01.849 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:01.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f03c406a050 msgr2=0x7f03c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f03c406a050 0x7f03c406c510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03c8036000 tx=0x7f03c8008030 comp rx=0 tx=0).stop 2023-10-19T10:38:01.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 msgr2=0x7f03d80c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 0x7f03d80c6090 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00d800 tx=0x7f03cc00dcc0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 shutdown_connections 2023-10-19T10:38:01.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f03c406a050 0x7f03c406c510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03d80b7f10 0x7f03d80b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03d815c960 0x7f03d80c6090 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 --2- 172.21.15.50:0/3487542477 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03d80b4070 0x7f03d8150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.847+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 >> 172.21.15.50:0/3487542477 conn(0x7f03d809b970 msgr2=0x7f03d80b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.848+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 shutdown_connections 2023-10-19T10:38:01.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.848+0000 7f03de185640 1 -- 172.21.15.50:0/3487542477 wait complete. 2023-10-19T10:38:01.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:38:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.967+0000 7fdb95aee640 1 Processor -- start 2023-10-19T10:38:01.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.967+0000 7fdb95aee640 1 -- start start 2023-10-19T10:38:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb95aee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900b26e0 0x7fdb9006f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb95aee640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdb900ce730 0x7fdb9006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb95aee640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb900701b0 con 0x7fdb90145880 2023-10-19T10:38:01.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb95aee640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9006f910 con 0x7fdb900ce730 2023-10-19T10:38:01.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb95aee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9006fbf0 con 0x7fdb900b26e0 2023-10-19T10:38:01.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb8ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdb900ce730 0x7fdb9006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb8ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fdb900ce730 0x7fdb9006aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55554/0 (socket says 172.21.15.50:55554) 2023-10-19T10:38:01.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb8ffff640 1 -- 172.21.15.50:0/2104266413 learned_addr learned my addr 172.21.15.50:0/2104266413 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:01.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.968+0000 7fdb8f7fe640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900b26e0 0x7fdb9006f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8ffff640 1 -- 172.21.15.50:0/2104266413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900b26e0 msgr2=0x7fdb9006f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:01.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8ffff640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900b26e0 0x7fdb9006f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8ffff640 1 -- 172.21.15.50:0/2104266413 >> v1:172.21.15.183:6789/0 conn(0x7fdb90145880 legacy=0x7fdb900ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8ffff640 1 -- 172.21.15.50:0/2104266413 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9006fed0 con 0x7fdb900ce730 2023-10-19T10:38:01.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8f7fe640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900b26e0 0x7fdb9006f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:01.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8ffff640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900ce730 0x7fdb9006aed0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009e50 tx=0x7fdb8802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdb054b81078553 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8e7fc640 1 -- 172.21.15.50:0/2104266413 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8802ca70 con 0x7fdb900ce730 2023-10-19T10:38:01.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8e7fc640 1 -- 172.21.15.50:0/2104266413 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8802cbd0 con 0x7fdb900ce730 2023-10-19T10:38:01.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.969+0000 7fdb8e7fc640 1 -- 172.21.15.50:0/2104266413 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880343f0 con 0x7fdb900ce730 2023-10-19T10:38:01.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 -- 172.21.15.50:0/2104266413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900ce730 msgr2=0x7fdb9006aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900ce730 0x7fdb9006aed0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009e50 tx=0x7fdb8802bed0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 -- 172.21.15.50:0/2104266413 shutdown_connections 2023-10-19T10:38:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900ce730 0x7fdb9006aed0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 --2- 172.21.15.50:0/2104266413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900b26e0 0x7fdb9006f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 -- 172.21.15.50:0/2104266413 >> 172.21.15.50:0/2104266413 conn(0x7fdb900ccf60 msgr2=0x7fdb900cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 -- 172.21.15.50:0/2104266413 shutdown_connections 2023-10-19T10:38:01.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.970+0000 7fdb95aee640 1 -- 172.21.15.50:0/2104266413 wait complete. 2023-10-19T10:38:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.971+0000 7fdb95aee640 1 Processor -- start 2023-10-19T10:38:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.971+0000 7fdb95aee640 1 -- start start 2023-10-19T10:38:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb95aee640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 0x7fdb900b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb95aee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900ce730 0x7fdb90155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb95aee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb90145880 0x7fdb900c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb95aee640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b62d0 con 0x7fdb900b26e0 2023-10-19T10:38:01.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb95aee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b5650 con 0x7fdb90145880 2023-10-19T10:38:01.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb95aee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900b5930 con 0x7fdb900ce730 2023-10-19T10:38:01.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 0x7fdb900b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 0x7fdb900b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55564/0 (socket says 172.21.15.50:55564) 2023-10-19T10:38:01.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 -- 172.21.15.50:0/956532012 learned_addr learned my addr 172.21.15.50:0/956532012 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:01.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8ffff640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb90145880 0x7fdb900c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8effd640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900ce730 0x7fdb90155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 -- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900ce730 msgr2=0x7fdb90155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900ce730 0x7fdb90155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 -- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb90145880 msgr2=0x7fdb900c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:01.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb90145880 0x7fdb900c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:01.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8f7fe640 1 -- 172.21.15.50:0/956532012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb88009b30 con 0x7fdb900b26e0 2023-10-19T10:38:01.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.972+0000 7fdb8effd640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900ce730 0x7fdb90155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:01.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.973+0000 7fdb8f7fe640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 0x7fdb900b5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000d7d0 tx=0x7fdb8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.973+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80004030 con 0x7fdb900b26e0 2023-10-19T10:38:01.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.973+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb80004190 con 0x7fdb900b26e0 2023-10-19T10:38:01.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.973+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80004520 con 0x7fdb900b26e0 2023-10-19T10:38:01.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.973+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb900b5c10 con 0x7fdb900b26e0 2023-10-19T10:38:01.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.973+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb901563d0 con 0x7fdb900b26e0 2023-10-19T10:38:01.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.974+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb90157220 con 0x7fdb900b26e0 2023-10-19T10:38:01.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.974+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdb800096a0 con 0x7fdb900b26e0 2023-10-19T10:38:01.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.975+0000 7fdb77fff640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdb7c06a050 0x7fdb7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:01.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.975+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdb800889f0 con 0x7fdb900b26e0 2023-10-19T10:38:01.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.975+0000 7fdb8effd640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdb7c06a050 0x7fdb7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:01.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.976+0000 7fdb8effd640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdb7c06a050 0x7fdb7c06c510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb84007fd0 tx=0x7fdb84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:01.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:01.979+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fdb80053db0 con 0x7fdb900b26e0 2023-10-19T10:38:02.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.155+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb9004e420 con 0x7fdb900b26e0 2023-10-19T10:38:02.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.155+0000 7fdb77fff640 1 -- 172.21.15.50:0/956532012 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb800573d0 con 0x7fdb900b26e0 2023-10-19T10:38:02.158 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:38:02.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdb7c06a050 msgr2=0x7fdb7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdb7c06a050 0x7fdb7c06c510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb84007fd0 tx=0x7fdb84009040 comp rx=0 tx=0).stop 2023-10-19T10:38:02.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 msgr2=0x7fdb900b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 0x7fdb900b5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000d7d0 tx=0x7fdb8000dc90 comp rx=0 tx=0).stop 2023-10-19T10:38:02.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 shutdown_connections 2023-10-19T10:38:02.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdb7c06a050 0x7fdb7c06c510 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb90145880 0x7fdb900c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb900ce730 0x7fdb90155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 --2- 172.21.15.50:0/956532012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb900b26e0 0x7fdb900b5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 >> 172.21.15.50:0/956532012 conn(0x7fdb900ccf60 msgr2=0x7fdb9015d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:02.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 shutdown_connections 2023-10-19T10:38:02.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.157+0000 7fdb95aee640 1 -- 172.21.15.50:0/956532012 wait complete. 2023-10-19T10:38:02.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:02.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.278+0000 7f210c385640 1 Processor -- start 2023-10-19T10:38:02.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.278+0000 7f210c385640 1 -- start start 2023-10-19T10:38:02.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.278+0000 7f210c385640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2104161210 0x7f21040d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.278+0000 7f210c385640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21040af250 con 0x7f21040b38f0 2023-10-19T10:38:02.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210c385640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21040af510 con 0x7f21040b3090 2023-10-19T10:38:02.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210c385640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21040b0190 con 0x7f2104161210 2023-10-19T10:38:02.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210a121640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f21040b38f0 0x7f2104160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46832/0 (socket says 172.21.15.50:46832) 2023-10-19T10:38:02.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210a121640 1 -- 172.21.15.50:0/3532566185 learned_addr learned my addr 172.21.15.50:0/3532566185 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:02.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210a922640 1 --2- 172.21.15.50:0/3532566185 >> v2:172.21.15.183:3300/0 conn(0x7f2104161210 0x7f21040d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356488811 0 0) 0x7f21040af250 con 0x7f21040b38f0 2023-10-19T10:38:02.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ec003520 con 0x7f21040b38f0 2023-10-19T10:38:02.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 610416800 0 0) 0x7f20ec003520 con 0x7f21040b38f0 2023-10-19T10:38:02.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 >> v2:172.21.15.183:3300/0 conn(0x7f2104161210 msgr2=0x7f21040d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:02.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 --2- 172.21.15.50:0/3532566185 >> v2:172.21.15.183:3300/0 conn(0x7f2104161210 0x7f21040d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 >> v1:172.21.15.183:6789/0 conn(0x7f21040b3090 legacy=0x7f21040b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21040af7f0 con 0x7f21040b38f0 2023-10-19T10:38:02.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210a922640 1 --2- 172.21.15.50:0/3532566185 >> v2:172.21.15.183:3300/0 conn(0x7f2104161210 0x7f21040d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:02.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f20fc001b70 con 0x7f21040b38f0 2023-10-19T10:38:02.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 >> v1:172.21.15.50:6789/0 conn(0x7f21040b38f0 legacy=0x7f2104160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec003970 0x7f20ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ec0047c0 0x7f20ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20ec005180 0x7f20ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21040b0190 con 0x7f20ec0047c0 2023-10-19T10:38:02.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec00c020 con 0x7f20ec003970 2023-10-19T10:38:02.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec00c1e0 con 0x7f20ec005180 2023-10-19T10:38:02.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f2109920640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ec0047c0 0x7f20ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.279+0000 7f210a121640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec003970 0x7f20ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f210a922640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20ec005180 0x7f20ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.289 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f2109920640 1 -- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20ec005180 msgr2=0x7f20ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:02.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f2109920640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f20ec005180 0x7f20ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f2109920640 1 -- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec003970 msgr2=0x7f20ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:02.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f2109920640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec003970 0x7f20ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f210a922640 1 -- 172.21.15.50:0/3532566185 reap_dead start 2023-10-19T10:38:02.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f2109920640 1 -- 172.21.15.50:0/3532566185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec003520 con 0x7f20ec0047c0 2023-10-19T10:38:02.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.280+0000 7f2109920640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ec0047c0 0x7f20ec004c40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f20f8001750 tx=0x7f20ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce66d74849f02f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:02.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80052d0 con 0x7f20ec0047c0 2023-10-19T10:38:02.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f802e050 con 0x7f20ec0047c0 2023-10-19T10:38:02.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210911f640 1 -- 172.21.15.50:0/3532566185 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8003830 con 0x7f20ec0047c0 2023-10-19T10:38:02.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 -- 172.21.15.50:0/3532566185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ec0047c0 msgr2=0x7f20ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ec0047c0 0x7f20ec004c40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f20f8001750 tx=0x7f20ec0057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 -- 172.21.15.50:0/3532566185 shutdown_connections 2023-10-19T10:38:02.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 --2- 172.21.15.50:0/3532566185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ec0047c0 0x7f20ec004c40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 -- 172.21.15.50:0/3532566185 >> 172.21.15.50:0/3532566185 conn(0x7f210409b960 msgr2=0x7f21040d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:02.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 -- 172.21.15.50:0/3532566185 shutdown_connections 2023-10-19T10:38:02.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.281+0000 7f210c385640 1 -- 172.21.15.50:0/3532566185 wait complete. 2023-10-19T10:38:02.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.282+0000 7f210c385640 1 Processor -- start 2023-10-19T10:38:02.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.282+0000 7f210c385640 1 -- start start 2023-10-19T10:38:02.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210c385640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec0047c0 0x7f21040ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210c385640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 0x7f210415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210c385640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21040c77e0 0x7f21040c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210c385640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21040c81a0 con 0x7f21040c9800 2023-10-19T10:38:02.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210c385640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21040cf0e0 con 0x7f20ec0047c0 2023-10-19T10:38:02.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210c385640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21040c9dc0 con 0x7f21040c77e0 2023-10-19T10:38:02.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f2109920640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 0x7f210415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f2109920640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 0x7f210415c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55570/0 (socket says 172.21.15.50:55570) 2023-10-19T10:38:02.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f2109920640 1 -- 172.21.15.50:0/2893437388 learned_addr learned my addr 172.21.15.50:0/2893437388 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:02.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210a121640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec0047c0 0x7f21040ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f210a922640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21040c77e0 0x7f21040c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f2109920640 1 -- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21040c77e0 msgr2=0x7f21040c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f2109920640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21040c77e0 0x7f21040c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.283+0000 7f2109920640 1 -- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec0047c0 msgr2=0x7f21040ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f2109920640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec0047c0 0x7f21040ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f2109920640 1 -- 172.21.15.50:0/2893437388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21040ca0a0 con 0x7f21040c9800 2023-10-19T10:38:02.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f210a121640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec0047c0 0x7f21040ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:02.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f2109920640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 0x7f210415c100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f20f802bc00 tx=0x7f20f8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:02.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80076c0 con 0x7f21040c9800 2023-10-19T10:38:02.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f8007a30 con 0x7f21040c9800 2023-10-19T10:38:02.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21040ca380 con 0x7f21040c9800 2023-10-19T10:38:02.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8006be0 con 0x7f21040c9800 2023-10-19T10:38:02.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.284+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21040ca660 con 0x7f21040c9800 2023-10-19T10:38:02.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.285+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210404e420 con 0x7f21040c9800 2023-10-19T10:38:02.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.285+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f20f8006d40 con 0x7f21040c9800 2023-10-19T10:38:02.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.286+0000 7f2102ffd640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f20f006a080 0x7f20f006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.286+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f20f80aa480 con 0x7f21040c9800 2023-10-19T10:38:02.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.286+0000 7f210a121640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f20f006a080 0x7f20f006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.287+0000 7f210a121640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f20f006a080 0x7f20f006c540 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20fc001550 tx=0x7f20fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:02.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.290+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f20f8079360 con 0x7f21040c9800 2023-10-19T10:38:02.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.463+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f21040b86b0 con 0x7f21040c9800 2023-10-19T10:38:02.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.464+0000 7f2102ffd640 1 -- 172.21.15.50:0/2893437388 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f20f8078ef0 con 0x7f21040c9800 2023-10-19T10:38:02.467 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:02.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f20f006a080 msgr2=0x7f20f006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f20f006a080 0x7f20f006c540 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20fc001550 tx=0x7f20fc009040 comp rx=0 tx=0).stop 2023-10-19T10:38:02.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 msgr2=0x7f210415c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 0x7f210415c100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f20f802bc00 tx=0x7f20f8001120 comp rx=0 tx=0).stop 2023-10-19T10:38:02.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 shutdown_connections 2023-10-19T10:38:02.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f20f006a080 0x7f20f006c540 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21040c77e0 0x7f21040c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21040c9800 0x7f210415c100 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 --2- 172.21.15.50:0/2893437388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ec0047c0 0x7f21040ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 >> 172.21.15.50:0/2893437388 conn(0x7f210409b960 msgr2=0x7f21040b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:02.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 shutdown_connections 2023-10-19T10:38:02.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.466+0000 7f210c385640 1 -- 172.21.15.50:0/2893437388 wait complete. 2023-10-19T10:38:02.479 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:38:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.584+0000 7fc63a0e9640 1 Processor -- start 2023-10-19T10:38:02.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.584+0000 7fc63a0e9640 1 -- start start 2023-10-19T10:38:02.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc63a0e9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc63415c570 0x7fc6340d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc63a0e9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc63415c950 0x7fc6340cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc63a0e9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6340cde60 con 0x7fc63406b760 2023-10-19T10:38:02.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc63a0e9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63406aa30 con 0x7fc63415c950 2023-10-19T10:38:02.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc63a0e9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63406b2d0 con 0x7fc63415c570 2023-10-19T10:38:02.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc63415c950 0x7fc6340cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc63415c950 0x7fc6340cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55580/0 (socket says 172.21.15.50:55580) 2023-10-19T10:38:02.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 -- 172.21.15.50:0/214449213 learned_addr learned my addr 172.21.15.50:0/214449213 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:02.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc6337fe640 1 --2- 172.21.15.50:0/214449213 >> v2:172.21.15.183:3300/0 conn(0x7fc63415c570 0x7fc6340d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 -- 172.21.15.50:0/214449213 >> v2:172.21.15.183:3300/0 conn(0x7fc63415c570 msgr2=0x7fc6340d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:02.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 --2- 172.21.15.50:0/214449213 >> v2:172.21.15.183:3300/0 conn(0x7fc63415c570 0x7fc6340d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 -- 172.21.15.50:0/214449213 >> v1:172.21.15.183:6789/0 conn(0x7fc63406b760 legacy=0x7fc6340d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc632ffd640 1 -- 172.21.15.50:0/214449213 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6340ce090 con 0x7fc63415c950 2023-10-19T10:38:02.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.585+0000 7fc6337fe640 1 --2- 172.21.15.50:0/214449213 >> v2:172.21.15.183:3300/0 conn(0x7fc63415c570 0x7fc6340d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:02.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.586+0000 7fc632ffd640 1 --2- 172.21.15.50:0/214449213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c950 0x7fc6340cd920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6340d2670 tx=0x7fc62802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec40d4e177b0b228 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:02.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.586+0000 7fc6327fc640 1 -- 172.21.15.50:0/214449213 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628031870 con 0x7fc63415c950 2023-10-19T10:38:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.586+0000 7fc6327fc640 1 -- 172.21.15.50:0/214449213 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6280319d0 con 0x7fc63415c950 2023-10-19T10:38:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.586+0000 7fc6327fc640 1 -- 172.21.15.50:0/214449213 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628031d60 con 0x7fc63415c950 2023-10-19T10:38:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 -- 172.21.15.50:0/214449213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c950 msgr2=0x7fc6340cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/214449213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c950 0x7fc6340cd920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6340d2670 tx=0x7fc62802b910 comp rx=0 tx=0).stop 2023-10-19T10:38:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 -- 172.21.15.50:0/214449213 shutdown_connections 2023-10-19T10:38:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/214449213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c950 0x7fc6340cd920 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/214449213 >> v2:172.21.15.183:3300/0 conn(0x7fc63415c570 0x7fc6340d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 -- 172.21.15.50:0/214449213 >> 172.21.15.50:0/214449213 conn(0x7fc634089ff0 msgr2=0x7fc63409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 -- 172.21.15.50:0/214449213 shutdown_connections 2023-10-19T10:38:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.587+0000 7fc63a0e9640 1 -- 172.21.15.50:0/214449213 wait complete. 2023-10-19T10:38:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.588+0000 7fc63a0e9640 1 Processor -- start 2023-10-19T10:38:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.588+0000 7fc63a0e9640 1 -- start start 2023-10-19T10:38:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc63a0e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc63406b760 0x7fc63415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc63a0e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 0x7fc634157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc63a0e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63415c950 0x7fc6340c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc63a0e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340c6350 con 0x7fc63415c570 2023-10-19T10:38:02.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc63a0e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63415e280 con 0x7fc63406b760 2023-10-19T10:38:02.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc63a0e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63415e560 con 0x7fc63415c950 2023-10-19T10:38:02.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 0x7fc634157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 0x7fc634157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55596/0 (socket says 172.21.15.50:55596) 2023-10-19T10:38:02.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 -- 172.21.15.50:0/1944137785 learned_addr learned my addr 172.21.15.50:0/1944137785 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:02.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc6337fe640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc63406b760 0x7fc63415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc633fff640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63415c950 0x7fc6340c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 -- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63415c950 msgr2=0x7fc6340c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63415c950 0x7fc6340c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 -- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc63406b760 msgr2=0x7fc63415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:02.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc63406b760 0x7fc63415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:02.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.589+0000 7fc632ffd640 1 -- 172.21.15.50:0/1944137785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280094d0 con 0x7fc63415c570 2023-10-19T10:38:02.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.590+0000 7fc632ffd640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 0x7fc634157ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc62802b950 tx=0x7fc6280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:02.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.590+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62803ad30 con 0x7fc63415c570 2023-10-19T10:38:02.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.590+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6280443f0 con 0x7fc63415c570 2023-10-19T10:38:02.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.590+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63415e840 con 0x7fc63415c570 2023-10-19T10:38:02.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.590+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6280447a0 con 0x7fc63415c570 2023-10-19T10:38:02.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.590+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63415ee00 con 0x7fc63415c570 2023-10-19T10:38:02.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.591+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63415eff0 con 0x7fc63415c570 2023-10-19T10:38:02.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.591+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc62804d450 con 0x7fc63415c570 2023-10-19T10:38:02.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.592+0000 7fc61bfff640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc62006a050 0x7fc62006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:02.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.592+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc62802d040 con 0x7fc63415c570 2023-10-19T10:38:02.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.592+0000 7fc6337fe640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc62006a050 0x7fc62006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:02.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.592+0000 7fc6337fe640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc62006a050 0x7fc62006c510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc624009930 tx=0x7fc624008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:02.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.596+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc6280761e0 con 0x7fc63415c570 2023-10-19T10:38:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.783+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc634073990 con 0x7fc63415c570 2023-10-19T10:38:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.783+0000 7fc61bfff640 1 -- 172.21.15.50:0/1944137785 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc628035020 con 0x7fc63415c570 2023-10-19T10:38:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:38:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc62006a050 msgr2=0x7fc62006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc62006a050 0x7fc62006c510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc624009930 tx=0x7fc624008040 comp rx=0 tx=0).stop 2023-10-19T10:38:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 msgr2=0x7fc634157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 0x7fc634157ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc62802b950 tx=0x7fc6280046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 shutdown_connections 2023-10-19T10:38:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc62006a050 0x7fc62006c510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63415c950 0x7fc6340c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc63415c570 0x7fc634157ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 --2- 172.21.15.50:0/1944137785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc63406b760 0x7fc63415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 >> 172.21.15.50:0/1944137785 conn(0x7fc634089ff0 msgr2=0x7fc6340cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 shutdown_connections 2023-10-19T10:38:03.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:02.785+0000 7fc63a0e9640 1 -- 172.21.15.50:0/1944137785 wait complete. 2023-10-19T10:38:03.048 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi183.front.sepia.ceph.com... 2023-10-19T10:38:03.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:03.049 DEBUG:teuthology.orchestra.run.smithi183:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:03.083 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:03.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:38:03.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:03.085 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-19T10:38:03.160 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:38:03.160 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:03.178 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:38:03.178 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:03.236 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:03.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.386+0000 7fd6179eb640 1 Processor -- start 2023-10-19T10:38:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.386+0000 7fd6179eb640 1 -- start start 2023-10-19T10:38:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd6179eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd61015c570 0x7fd6100d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd6179eb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd61015c950 0x7fd6100cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd6179eb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6100cde60 con 0x7fd61006b760 2023-10-19T10:38:03.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd6179eb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61006aa30 con 0x7fd61015c950 2023-10-19T10:38:03.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd6179eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61006b2d0 con 0x7fd61015c570 2023-10-19T10:38:03.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd615f88640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd61006b760 0x7fd6100d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46846/0 (socket says 172.21.15.50:46846) 2023-10-19T10:38:03.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd615f88640 1 -- 172.21.15.50:0/2883212828 learned_addr learned my addr 172.21.15.50:0/2883212828 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:03.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd615787640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd61015c570 0x7fd6100d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd614f86640 1 --2- 172.21.15.50:0/2883212828 >> v2:172.21.15.183:3300/0 conn(0x7fd61015c950 0x7fd6100cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216760098 0 0) 0x7fd6100cde60 con 0x7fd61006b760 2023-10-19T10:38:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5fc003520 con 0x7fd61006b760 2023-10-19T10:38:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965564860 0 0) 0x7fd5fc003520 con 0x7fd61006b760 2023-10-19T10:38:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd61015c570 msgr2=0x7fd6100d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd61015c570 0x7fd6100d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 >> v2:172.21.15.183:3300/0 conn(0x7fd61015c950 msgr2=0x7fd6100cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 --2- 172.21.15.50:0/2883212828 >> v2:172.21.15.183:3300/0 conn(0x7fd61015c950 0x7fd6100cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100ce090 con 0x7fd61006b760 2023-10-19T10:38:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.387+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fd6080019f0 con 0x7fd61006b760 2023-10-19T10:38:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 >> v1:172.21.15.50:6789/0 conn(0x7fd61006b760 legacy=0x7fd6100d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5fc003970 0x7fd5fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc004360 0x7fd5fc004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5fc004f50 0x7fd5fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61006aa30 con 0x7fd5fc004f50 2023-10-19T10:38:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61006b2d0 con 0x7fd5fc004360 2023-10-19T10:38:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc00bef0 con 0x7fd5fc003970 2023-10-19T10:38:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615787640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd61015c570 0x7fd6100d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd614f86640 1 --2- 172.21.15.50:0/2883212828 >> v2:172.21.15.183:3300/0 conn(0x7fd61015c950 0x7fd6100cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615f88640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5fc004f50 0x7fd5fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd614f86640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc004360 0x7fd5fc004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615787640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5fc003970 0x7fd5fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615f88640 1 -- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5fc003970 msgr2=0x7fd5fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615f88640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5fc003970 0x7fd5fc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615f88640 1 -- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc004360 msgr2=0x7fd5fc004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615f88640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc004360 0x7fd5fc004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.388+0000 7fd615f88640 1 -- 172.21.15.50:0/2883212828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc003520 con 0x7fd5fc004f50 2023-10-19T10:38:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd615f88640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5fc004f50 0x7fd5fc00b980 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd60800a860 tx=0x7fd5fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c442101609574229 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608034630 con 0x7fd5fc004f50 2023-10-19T10:38:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60802f070 con 0x7fd5fc004f50 2023-10-19T10:38:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd607fff640 1 -- 172.21.15.50:0/2883212828 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608034a30 con 0x7fd5fc004f50 2023-10-19T10:38:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd6179eb640 1 -- 172.21.15.50:0/2883212828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5fc004f50 msgr2=0x7fd5fc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd6179eb640 1 --2- 172.21.15.50:0/2883212828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5fc004f50 0x7fd5fc00b980 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd60800a860 tx=0x7fd5fc0057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd615f88640 1 -- 172.21.15.50:0/2883212828 reap_dead start 2023-10-19T10:38:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd6179eb640 1 -- 172.21.15.50:0/2883212828 shutdown_connections 2023-10-19T10:38:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.389+0000 7fd6179eb640 1 -- 172.21.15.50:0/2883212828 >> 172.21.15.50:0/2883212828 conn(0x7fd610089ff0 msgr2=0x7fd61009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.390+0000 7fd6179eb640 1 -- 172.21.15.50:0/2883212828 shutdown_connections 2023-10-19T10:38:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.390+0000 7fd6179eb640 1 -- 172.21.15.50:0/2883212828 wait complete. 2023-10-19T10:38:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 Processor -- start 2023-10-19T10:38:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 -- start start 2023-10-19T10:38:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 0x7fd610159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6100c91c0 0x7fd61015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6100cb370 0x7fd6100cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61015a6b0 con 0x7fd6100c8de0 2023-10-19T10:38:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100c9910 con 0x7fd6100cb370 2023-10-19T10:38:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.391+0000 7fd6179eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100c9bf0 con 0x7fd6100c91c0 2023-10-19T10:38:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 0x7fd610159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 0x7fd610159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55622/0 (socket says 172.21.15.50:55622) 2023-10-19T10:38:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 -- 172.21.15.50:0/3368447651 learned_addr learned my addr 172.21.15.50:0/3368447651 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615f88640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6100cb370 0x7fd6100cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd614f86640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6100c91c0 0x7fd61015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 -- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6100c91c0 msgr2=0x7fd61015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6100c91c0 0x7fd61015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 -- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6100cb370 msgr2=0x7fd6100cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6100cb370 0x7fd6100cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 -- 172.21.15.50:0/3368447651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100c9ed0 con 0x7fd6100c8de0 2023-10-19T10:38:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd615787640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 0x7fd610159c30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd60c011800 tx=0x7fd60c011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00f9c0 con 0x7fd6100c8de0 2023-10-19T10:38:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00fb20 con 0x7fd6100c8de0 2023-10-19T10:38:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6100ca1b0 con 0x7fd6100c8de0 2023-10-19T10:38:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.392+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00feb0 con 0x7fd6100c8de0 2023-10-19T10:38:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.393+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6100ca6d0 con 0x7fd6100c8de0 2023-10-19T10:38:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.393+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd610073990 con 0x7fd6100c8de0 2023-10-19T10:38:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.394+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd60c015460 con 0x7fd6100c8de0 2023-10-19T10:38:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.394+0000 7fd605ffb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd5f406a050 0x7fd5f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.394+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd60c089020 con 0x7fd6100c8de0 2023-10-19T10:38:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.395+0000 7fd614f86640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd5f406a050 0x7fd5f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.395+0000 7fd614f86640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd5f406a050 0x7fd5f406c510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6100c8b00 tx=0x7fd600008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:03.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.399+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd60c054470 con 0x7fd6100c8de0 2023-10-19T10:38:03.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.576+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6100ca370 con 0x7fd6100c8de0 2023-10-19T10:38:03.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.577+0000 7fd605ffb640 1 -- 172.21.15.50:0/3368447651 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fd60c057a90 con 0x7fd6100c8de0 2023-10-19T10:38:03.580 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:03.580 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:03.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd5f406a050 msgr2=0x7fd5f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd5f406a050 0x7fd5f406c510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6100c8b00 tx=0x7fd600008040 comp rx=0 tx=0).stop 2023-10-19T10:38:03.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 msgr2=0x7fd610159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 0x7fd610159c30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd60c011800 tx=0x7fd60c011c00 comp rx=0 tx=0).stop 2023-10-19T10:38:03.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 shutdown_connections 2023-10-19T10:38:03.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd5f406a050 0x7fd5f406c510 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6100cb370 0x7fd6100cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6100c91c0 0x7fd61015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 --2- 172.21.15.50:0/3368447651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6100c8de0 0x7fd610159c30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 >> 172.21.15.50:0/3368447651 conn(0x7fd610089ff0 msgr2=0x7fd6100b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:03.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 shutdown_connections 2023-10-19T10:38:03.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.579+0000 7fd6179eb640 1 -- 172.21.15.50:0/3368447651 wait complete. 2023-10-19T10:38:03.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:38:03.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:03.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.700+0000 7faed87cb640 1 Processor -- start 2023-10-19T10:38:03.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.700+0000 7faed87cb640 1 -- start start 2023-10-19T10:38:03.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed87cb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faed00b3090 0x7faed00d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed87cb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faed0161210 0x7faed00b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed87cb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00af250 con 0x7faed00b38f0 2023-10-19T10:38:03.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed87cb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00b0190 con 0x7faed0161210 2023-10-19T10:38:03.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed87cb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00af510 con 0x7faed00b3090 2023-10-19T10:38:03.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faed0161210 0x7faed00b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faed0161210 0x7faed00b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55634/0 (socket says 172.21.15.50:55634) 2023-10-19T10:38:03.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 -- 172.21.15.50:0/2125373244 learned_addr learned my addr 172.21.15.50:0/2125373244 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:03.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed5d66640 1 --2- 172.21.15.50:0/2125373244 >> v2:172.21.15.183:3300/0 conn(0x7faed00b3090 0x7faed00d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 -- 172.21.15.50:0/2125373244 >> v2:172.21.15.183:3300/0 conn(0x7faed00b3090 msgr2=0x7faed00d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:03.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 --2- 172.21.15.50:0/2125373244 >> v2:172.21.15.183:3300/0 conn(0x7faed00b3090 0x7faed00d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 -- 172.21.15.50:0/2125373244 >> v1:172.21.15.183:6789/0 conn(0x7faed00b38f0 legacy=0x7faed0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.701+0000 7faed6d68640 1 -- 172.21.15.50:0/2125373244 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00af7f0 con 0x7faed0161210 2023-10-19T10:38:03.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.702+0000 7faed6d68640 1 --2- 172.21.15.50:0/2125373244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed0161210 0x7faed00b05b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faed00c6230 tx=0x7faec002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d63219a38fd898 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:03.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.702+0000 7faed5565640 1 -- 172.21.15.50:0/2125373244 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec002cb20 con 0x7faed0161210 2023-10-19T10:38:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.702+0000 7faed5565640 1 -- 172.21.15.50:0/2125373244 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec002cc80 con 0x7faed0161210 2023-10-19T10:38:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.702+0000 7faed5565640 1 -- 172.21.15.50:0/2125373244 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00343f0 con 0x7faed0161210 2023-10-19T10:38:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 -- 172.21.15.50:0/2125373244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed0161210 msgr2=0x7faed00b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 --2- 172.21.15.50:0/2125373244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed0161210 0x7faed00b05b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faed00c6230 tx=0x7faec002bf80 comp rx=0 tx=0).stop 2023-10-19T10:38:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 -- 172.21.15.50:0/2125373244 shutdown_connections 2023-10-19T10:38:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 --2- 172.21.15.50:0/2125373244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed0161210 0x7faed00b05b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 --2- 172.21.15.50:0/2125373244 >> v2:172.21.15.183:3300/0 conn(0x7faed00b3090 0x7faed00d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 -- 172.21.15.50:0/2125373244 >> 172.21.15.50:0/2125373244 conn(0x7faed009b960 msgr2=0x7faed00d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 -- 172.21.15.50:0/2125373244 shutdown_connections 2023-10-19T10:38:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.703+0000 7faed87cb640 1 -- 172.21.15.50:0/2125373244 wait complete. 2023-10-19T10:38:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.704+0000 7faed87cb640 1 Processor -- start 2023-10-19T10:38:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.704+0000 7faed87cb640 1 -- start start 2023-10-19T10:38:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed87cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 0x7faed00c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed87cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed00b38f0 0x7faed0155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed87cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed0161210 0x7faed015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed87cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed01f27e0 con 0x7faed00b3090 2023-10-19T10:38:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed87cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed014a250 con 0x7faed0161210 2023-10-19T10:38:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed87cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed0156a40 con 0x7faed00b38f0 2023-10-19T10:38:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 0x7faed00c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 0x7faed00c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55642/0 (socket says 172.21.15.50:55642) 2023-10-19T10:38:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 -- 172.21.15.50:0/2660406811 learned_addr learned my addr 172.21.15.50:0/2660406811 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:03.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6d68640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed0161210 0x7faed015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed5d66640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed00b38f0 0x7faed0155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 -- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed00b38f0 msgr2=0x7faed0155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed00b38f0 0x7faed0155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 -- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed0161210 msgr2=0x7faed015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed0161210 0x7faed015d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.705+0000 7faed6567640 1 -- 172.21.15.50:0/2660406811 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0009b30 con 0x7faed00b3090 2023-10-19T10:38:03.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.706+0000 7faed6567640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 0x7faed00c8c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faec800c480 tx=0x7faec800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:03.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.706+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec80052d0 con 0x7faed00b3090 2023-10-19T10:38:03.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.706+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec8005450 con 0x7faed00b3090 2023-10-19T10:38:03.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.706+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed0156d20 con 0x7faed00b3090 2023-10-19T10:38:03.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.706+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8003880 con 0x7faed00b3090 2023-10-19T10:38:03.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.706+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01572e0 con 0x7faed00b3090 2023-10-19T10:38:03.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.707+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed01f2950 con 0x7faed00b3090 2023-10-19T10:38:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.707+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faec80039e0 con 0x7faed00b3090 2023-10-19T10:38:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.708+0000 7faeceffd640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a080 0x7faebc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:03.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.708+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faec80883b0 con 0x7faed00b3090 2023-10-19T10:38:03.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.708+0000 7faed5d66640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a080 0x7faebc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:03.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.708+0000 7faed5d66640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a080 0x7faebc06c540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faed00cb970 tx=0x7faec4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:03.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.712+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7faed01f2950 con 0x7faed00b3090 2023-10-19T10:38:03.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.886+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faed01f2920 con 0x7faed00b3090 2023-10-19T10:38:03.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.887+0000 7faeceffd640 1 -- 172.21.15.50:0/2660406811 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7faec8053810 con 0x7faed00b3090 2023-10-19T10:38:03.890 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:03.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a080 msgr2=0x7faebc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a080 0x7faebc06c540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faed00cb970 tx=0x7faec4009190 comp rx=0 tx=0).stop 2023-10-19T10:38:03.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 msgr2=0x7faed00c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:03.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 0x7faed00c8c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faec800c480 tx=0x7faec800c940 comp rx=0 tx=0).stop 2023-10-19T10:38:03.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 shutdown_connections 2023-10-19T10:38:03.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a080 0x7faebc06c540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed0161210 0x7faed015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed00b38f0 0x7faed0155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 --2- 172.21.15.50:0/2660406811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3090 0x7faed00c8c40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:03.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 >> 172.21.15.50:0/2660406811 conn(0x7faed009b960 msgr2=0x7faed00cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:03.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 shutdown_connections 2023-10-19T10:38:03.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:03.889+0000 7faed87cb640 1 -- 172.21.15.50:0/2660406811 wait complete. 2023-10-19T10:38:03.902 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:04.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.009+0000 7f96a4641640 1 Processor -- start 2023-10-19T10:38:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.009+0000 7f96a4641640 1 -- start start 2023-10-19T10:38:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a4641640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f969c158a10 0x7f969c070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a4641640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a4641640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0cc0f0 con 0x7f969c06bdf0 2023-10-19T10:38:04.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a4641640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c158df0 con 0x7f969c158a10 2023-10-19T10:38:04.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a4641640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c070a80 con 0x7f969c071130 2023-10-19T10:38:04.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f969c158a10 0x7f969c070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f969c158a10 0x7f969c070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55654/0 (socket says 172.21.15.50:55654) 2023-10-19T10:38:04.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 -- 172.21.15.50:0/2574729343 learned_addr learned my addr 172.21.15.50:0/2574729343 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a1bdc640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 -- 172.21.15.50:0/2574729343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 msgr2=0x7f969c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 -- 172.21.15.50:0/2574729343 >> v1:172.21.15.183:6789/0 conn(0x7f969c06bdf0 legacy=0x7f969c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a23dd640 1 -- 172.21.15.50:0/2574729343 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c070d50 con 0x7f969c158a10 2023-10-19T10:38:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.010+0000 7f96a1bdc640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.011+0000 7f96a23dd640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c158a10 0x7f969c070540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f968c009870 tx=0x7f968c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f894060319e116fb server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.011+0000 7f96a13db640 1 -- 172.21.15.50:0/2574729343 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c031640 con 0x7f969c158a10 2023-10-19T10:38:04.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.011+0000 7f96a13db640 1 -- 172.21.15.50:0/2574729343 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c0317a0 con 0x7f969c158a10 2023-10-19T10:38:04.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a13db640 1 -- 172.21.15.50:0/2574729343 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c031b30 con 0x7f969c158a10 2023-10-19T10:38:04.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 -- 172.21.15.50:0/2574729343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c158a10 msgr2=0x7f969c070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c158a10 0x7f969c070540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f968c009870 tx=0x7f968c02b8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 -- 172.21.15.50:0/2574729343 shutdown_connections 2023-10-19T10:38:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 --2- 172.21.15.50:0/2574729343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c158a10 0x7f969c070540 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 -- 172.21.15.50:0/2574729343 >> 172.21.15.50:0/2574729343 conn(0x7f969c0afbe0 msgr2=0x7f969c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 -- 172.21.15.50:0/2574729343 shutdown_connections 2023-10-19T10:38:04.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.012+0000 7f96a4641640 1 -- 172.21.15.50:0/2574729343 wait complete. 2023-10-19T10:38:04.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.013+0000 7f96a4641640 1 Processor -- start 2023-10-19T10:38:04.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.013+0000 7f96a4641640 1 -- start start 2023-10-19T10:38:04.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a4641640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 0x7f969c0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a4641640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c0c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a4641640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f969c158a10 0x7f969c0bd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a4641640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c14bc90 con 0x7f969c06bdf0 2023-10-19T10:38:04.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a4641640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c14b080 con 0x7f969c158a10 2023-10-19T10:38:04.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a4641640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c14b360 con 0x7f969c071130 2023-10-19T10:38:04.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 0x7f969c0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 0x7f969c0c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55662/0 (socket says 172.21.15.50:55662) 2023-10-19T10:38:04.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 -- 172.21.15.50:0/2585670618 learned_addr learned my addr 172.21.15.50:0/2585670618 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a1bdc640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c0c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a2bde640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f969c158a10 0x7f969c0bd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 -- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 msgr2=0x7f969c0c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c0c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 -- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f969c158a10 msgr2=0x7f969c0bd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.014+0000 7f96a23dd640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f969c158a10 0x7f969c0bd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f96a23dd640 1 -- 172.21.15.50:0/2585670618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c0094d0 con 0x7f969c06bdf0 2023-10-19T10:38:04.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f96a1bdc640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c0c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f96a2bde640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f969c158a10 0x7f969c0bd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f96a23dd640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 0x7f969c0c7900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f968c000c00 tx=0x7f968c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c03ac60 con 0x7f969c06bdf0 2023-10-19T10:38:04.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c03ade0 con 0x7f969c06bdf0 2023-10-19T10:38:04.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c14b640 con 0x7f969c06bdf0 2023-10-19T10:38:04.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c044630 con 0x7f969c06bdf0 2023-10-19T10:38:04.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.015+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0c7e40 con 0x7f969c06bdf0 2023-10-19T10:38:04.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.016+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c04e420 con 0x7f969c06bdf0 2023-10-19T10:38:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.016+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f968c02f050 con 0x7f969c06bdf0 2023-10-19T10:38:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.017+0000 7f9692ffd640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f968006a080 0x7f968006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.017+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f968c0aa990 con 0x7f969c06bdf0 2023-10-19T10:38:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.017+0000 7f96a1bdc640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f968006a080 0x7f968006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.032 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.017+0000 7f96a1bdc640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f968006a080 0x7f968006c540 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9698006fd0 tx=0x7f9698008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.032 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.021+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f968c075de0 con 0x7f969c06bdf0 2023-10-19T10:38:04.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.197+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f969c0cc5f0 con 0x7f969c06bdf0 2023-10-19T10:38:04.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.198+0000 7f9692ffd640 1 -- 172.21.15.50:0/2585670618 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f968c04d450 con 0x7f969c06bdf0 2023-10-19T10:38:04.201 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:04.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f968006a080 msgr2=0x7f968006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f968006a080 0x7f968006c540 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9698006fd0 tx=0x7f9698008040 comp rx=0 tx=0).stop 2023-10-19T10:38:04.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 msgr2=0x7f969c0c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 0x7f969c0c7900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f968c000c00 tx=0x7f968c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 shutdown_connections 2023-10-19T10:38:04.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f968006a080 0x7f968006c540 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f969c158a10 0x7f969c0bd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969c071130 0x7f969c0c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 --2- 172.21.15.50:0/2585670618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c06bdf0 0x7f969c0c7900 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 >> 172.21.15.50:0/2585670618 conn(0x7f969c0afbe0 msgr2=0x7f969c0adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 shutdown_connections 2023-10-19T10:38:04.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.200+0000 7f96a4641640 1 -- 172.21.15.50:0/2585670618 wait complete. 2023-10-19T10:38:04.214 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:38:04.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.320+0000 7f420533e640 1 Processor -- start 2023-10-19T10:38:04.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.320+0000 7f420533e640 1 -- start start 2023-10-19T10:38:04.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f420533e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42000b4070 0x7f42000b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f420533e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f420015c960 0x7f42000d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f420533e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000b4990 con 0x7f420015cdf0 2023-10-19T10:38:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f420533e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420015d280 con 0x7f420015c960 2023-10-19T10:38:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f420533e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420015d560 con 0x7f42000b4070 2023-10-19T10:38:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41fe7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f420015c960 0x7f42000d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41ff7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f420015cdf0 0x7f4200160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46852/0 (socket says 172.21.15.50:46852) 2023-10-19T10:38:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41ff7fe640 1 -- 172.21.15.50:0/180055962 learned_addr learned my addr 172.21.15.50:0/180055962 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41feffd640 1 --2- 172.21.15.50:0/180055962 >> v2:172.21.15.183:3300/0 conn(0x7f42000b4070 0x7f42000b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41fe7fc640 1 -- 172.21.15.50:0/180055962 >> v2:172.21.15.183:3300/0 conn(0x7f42000b4070 msgr2=0x7f42000b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41fe7fc640 1 --2- 172.21.15.50:0/180055962 >> v2:172.21.15.183:3300/0 conn(0x7f42000b4070 0x7f42000b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.321+0000 7f41fe7fc640 1 -- 172.21.15.50:0/180055962 >> v1:172.21.15.50:6789/0 conn(0x7f420015cdf0 legacy=0x7f4200160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.322+0000 7f41fe7fc640 1 -- 172.21.15.50:0/180055962 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420015d840 con 0x7f420015c960 2023-10-19T10:38:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.322+0000 7f41fdffb640 1 -- 172.21.15.50:0/180055962 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064930801 0 0) 0x7f42000b4990 con 0x7f420015cdf0 2023-10-19T10:38:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.322+0000 7f41fe7fc640 1 --2- 172.21.15.50:0/180055962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420015c960 0x7f42000d2d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f420008a3b0 tx=0x7f41e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3679338844687a22 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.322+0000 7f41fdffb640 1 -- 172.21.15.50:0/180055962 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8031830 con 0x7f420015c960 2023-10-19T10:38:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.322+0000 7f41fdffb640 1 -- 172.21.15.50:0/180055962 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8031990 con 0x7f420015c960 2023-10-19T10:38:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.322+0000 7f41fdffb640 1 -- 172.21.15.50:0/180055962 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8031d20 con 0x7f420015c960 2023-10-19T10:38:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 -- 172.21.15.50:0/180055962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420015c960 msgr2=0x7f42000d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 --2- 172.21.15.50:0/180055962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420015c960 0x7f42000d2d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f420008a3b0 tx=0x7f41e802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 -- 172.21.15.50:0/180055962 shutdown_connections 2023-10-19T10:38:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 --2- 172.21.15.50:0/180055962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420015c960 0x7f42000d2d70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 --2- 172.21.15.50:0/180055962 >> v2:172.21.15.183:3300/0 conn(0x7f42000b4070 0x7f42000b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 -- 172.21.15.50:0/180055962 >> 172.21.15.50:0/180055962 conn(0x7f420009b970 msgr2=0x7f42000d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 -- 172.21.15.50:0/180055962 shutdown_connections 2023-10-19T10:38:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.323+0000 7f420533e640 1 -- 172.21.15.50:0/180055962 wait complete. 2023-10-19T10:38:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.324+0000 7f420533e640 1 Processor -- start 2023-10-19T10:38:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.324+0000 7f420533e640 1 -- start start 2023-10-19T10:38:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f420533e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 0x7f42001e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f420533e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f420015c960 0x7f42001e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f420533e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4200155e60 0x7f420015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f420533e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42001e7dc0 con 0x7f42000b4070 2023-10-19T10:38:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f420533e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42001e80a0 con 0x7f4200155e60 2023-10-19T10:38:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f420533e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42001e8380 con 0x7f420015c960 2023-10-19T10:38:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 0x7f42001e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 0x7f42001e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55678/0 (socket says 172.21.15.50:55678) 2023-10-19T10:38:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 -- 172.21.15.50:0/1292081114 learned_addr learned my addr 172.21.15.50:0/1292081114 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41fe7fc640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f420015c960 0x7f42001e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41ff7fe640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4200155e60 0x7f420015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 -- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f420015c960 msgr2=0x7f42001e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f420015c960 0x7f42001e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 -- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4200155e60 msgr2=0x7f420015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4200155e60 0x7f420015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41feffd640 1 -- 172.21.15.50:0/1292081114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e8009400 con 0x7f42000b4070 2023-10-19T10:38:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.325+0000 7f41ff7fe640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4200155e60 0x7f420015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.326+0000 7f41feffd640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 0x7f42001e77e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f41f800d800 tx=0x7f41f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.326+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800b6c0 con 0x7f42000b4070 2023-10-19T10:38:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.326+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800b820 con 0x7f42000b4070 2023-10-19T10:38:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.326+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42000c6ca0 con 0x7f42000b4070 2023-10-19T10:38:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.326+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800bbd0 con 0x7f42000b4070 2023-10-19T10:38:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.326+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42001e73b0 con 0x7f42000b4070 2023-10-19T10:38:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.327+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420004e420 con 0x7f42000b4070 2023-10-19T10:38:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.327+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f41f800bd30 con 0x7f42000b4070 2023-10-19T10:38:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.328+0000 7f41f77fe640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f41e006a080 0x7f41e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.328+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f41f8088d90 con 0x7f42000b4070 2023-10-19T10:38:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.328+0000 7f41fe7fc640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f41e006a080 0x7f41e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.328+0000 7f41fe7fc640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f41e006a080 0x7f41e006c540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41e8036000 tx=0x7f41e8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.332+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f41f8057be0 con 0x7f42000b4070 2023-10-19T10:38:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.510+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42000ccb40 con 0x7f42000b4070 2023-10-19T10:38:04.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.510+0000 7f41f77fe640 1 -- 172.21.15.50:0/1292081114 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41f8057770 con 0x7f42000b4070 2023-10-19T10:38:04.513 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:04.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f41e006a080 msgr2=0x7f41e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f41e006a080 0x7f41e006c540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41e8036000 tx=0x7f41e8008030 comp rx=0 tx=0).stop 2023-10-19T10:38:04.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 msgr2=0x7f42001e77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 0x7f42001e77e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f41f800d800 tx=0x7f41f800dcc0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 shutdown_connections 2023-10-19T10:38:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f41e006a080 0x7f41e006c540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4200155e60 0x7f420015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f420015c960 0x7f42001e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 --2- 172.21.15.50:0/1292081114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42000b4070 0x7f42001e77e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 >> 172.21.15.50:0/1292081114 conn(0x7f420009b970 msgr2=0x7f420006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 shutdown_connections 2023-10-19T10:38:04.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.512+0000 7f420533e640 1 -- 172.21.15.50:0/1292081114 wait complete. 2023-10-19T10:38:04.525 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:38:04.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.632+0000 7f5d827d2640 1 Processor -- start 2023-10-19T10:38:04.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.632+0000 7f5d827d2640 1 -- start start 2023-10-19T10:38:04.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d827d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b3090 0x7f5d7c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d827d2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5d7c161210 0x7f5d7c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d827d2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c0af250 con 0x7f5d7c0b38f0 2023-10-19T10:38:04.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d827d2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0b0190 con 0x7f5d7c161210 2023-10-19T10:38:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d827d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0af510 con 0x7f5d7c0b3090 2023-10-19T10:38:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5d7c161210 0x7f5d7c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5d7c161210 0x7f5d7c0b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55692/0 (socket says 172.21.15.50:55692) 2023-10-19T10:38:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d7bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5d7c0b38f0 0x7f5d7c160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46854/0 (socket says 172.21.15.50:46854) 2023-10-19T10:38:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 -- 172.21.15.50:0/2007745153 learned_addr learned my addr 172.21.15.50:0/2007745153 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 -- 172.21.15.50:0/2007745153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b3090 msgr2=0x7f5d7c0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b3090 0x7f5d7c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d7affd640 1 -- 172.21.15.50:0/2007745153 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988966333 0 0) 0x7f5d7c0af250 con 0x7f5d7c0b38f0 2023-10-19T10:38:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b3090 0x7f5d7c0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 -- 172.21.15.50:0/2007745153 >> v1:172.21.15.50:6789/0 conn(0x7f5d7c0b38f0 legacy=0x7f5d7c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d80d6f640 1 -- 172.21.15.50:0/2007745153 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0af7f0 con 0x7f5d7c161210 2023-10-19T10:38:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.633+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b3090 0x7f5d7c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.634+0000 7f5d80d6f640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c0b05b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009e50 tx=0x7f5d7402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb342bea6f911236 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.634+0000 7f5d7affd640 1 -- 172.21.15.50:0/2007745153 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7402caf0 con 0x7f5d7c161210 2023-10-19T10:38:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.634+0000 7f5d7affd640 1 -- 172.21.15.50:0/2007745153 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7402cc50 con 0x7f5d7c161210 2023-10-19T10:38:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.634+0000 7f5d7affd640 1 -- 172.21.15.50:0/2007745153 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740343f0 con 0x7f5d7c161210 2023-10-19T10:38:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 -- 172.21.15.50:0/2007745153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 msgr2=0x7f5d7c0b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c0b05b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009e50 tx=0x7f5d7402bf50 comp rx=0 tx=0).stop 2023-10-19T10:38:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 -- 172.21.15.50:0/2007745153 shutdown_connections 2023-10-19T10:38:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c0b05b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 --2- 172.21.15.50:0/2007745153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b3090 0x7f5d7c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 -- 172.21.15.50:0/2007745153 >> 172.21.15.50:0/2007745153 conn(0x7f5d7c09b960 msgr2=0x7f5d7c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 -- 172.21.15.50:0/2007745153 shutdown_connections 2023-10-19T10:38:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.635+0000 7f5d827d2640 1 -- 172.21.15.50:0/2007745153 wait complete. 2023-10-19T10:38:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.636+0000 7f5d827d2640 1 Processor -- start 2023-10-19T10:38:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.636+0000 7f5d827d2640 1 -- start start 2023-10-19T10:38:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d827d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d7c0b3090 0x7f5d7c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d827d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b38f0 0x7f5d7c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d827d2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d827d2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0cab40 con 0x7f5d7c161210 2023-10-19T10:38:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d827d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c153c70 con 0x7f5d7c0b3090 2023-10-19T10:38:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d827d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ca530 con 0x7f5d7c0b38f0 2023-10-19T10:38:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d80d6f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d80d6f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55700/0 (socket says 172.21.15.50:55700) 2023-10-19T10:38:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d80d6f640 1 -- 172.21.15.50:0/1517993143 learned_addr learned my addr 172.21.15.50:0/1517993143 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b38f0 0x7f5d7c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d7c0b3090 0x7f5d7c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d80d6f640 1 -- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b38f0 msgr2=0x7f5d7c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.637+0000 7f5d80d6f640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b38f0 0x7f5d7c158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d80d6f640 1 -- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d7c0b3090 msgr2=0x7f5d7c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d80d6f640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d7c0b3090 0x7f5d7c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d80d6f640 1 -- 172.21.15.50:0/1517993143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74009b30 con 0x7f5d7c161210 2023-10-19T10:38:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b38f0 0x7f5d7c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d7c0b3090 0x7f5d7c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d80d6f640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c15af10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009e50 tx=0x7f5d74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74009510 con 0x7f5d7c161210 2023-10-19T10:38:04.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7403e450 con 0x7f5d7c161210 2023-10-19T10:38:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7403e600 con 0x7f5d7c161210 2023-10-19T10:38:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c1593b0 con 0x7f5d7c161210 2023-10-19T10:38:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.638+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c0ca7d0 con 0x7f5d7c161210 2023-10-19T10:38:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.639+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c0cacf0 con 0x7f5d7c161210 2023-10-19T10:38:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.639+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d7403e760 con 0x7f5d7c161210 2023-10-19T10:38:04.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.640+0000 7f5d78ff9640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d5c06a050 0x7f5d5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.640+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5d740ab380 con 0x7f5d7c161210 2023-10-19T10:38:04.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.640+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d5c06a050 0x7f5d5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.640+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d5c06a050 0x7f5d5c06c510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d64001020 tx=0x7f5d64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.644+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5d7c0cacf0 con 0x7f5d7c161210 2023-10-19T10:38:04.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.819+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d7c0afd80 con 0x7f5d7c161210 2023-10-19T10:38:04.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.819+0000 7f5d78ff9640 1 -- 172.21.15.50:0/1517993143 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d740767d0 con 0x7f5d7c161210 2023-10-19T10:38:04.822 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:38:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d5c06a050 msgr2=0x7f5d5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d5c06a050 0x7f5d5c06c510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d64001020 tx=0x7f5d64009040 comp rx=0 tx=0).stop 2023-10-19T10:38:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 msgr2=0x7f5d7c15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c15af10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009e50 tx=0x7f5d74004a40 comp rx=0 tx=0).stop 2023-10-19T10:38:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 shutdown_connections 2023-10-19T10:38:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d5c06a050 0x7f5d5c06c510 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c161210 0x7f5d7c15af10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d7c0b38f0 0x7f5d7c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 --2- 172.21.15.50:0/1517993143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d7c0b3090 0x7f5d7c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 >> 172.21.15.50:0/1517993143 conn(0x7f5d7c09b960 msgr2=0x7f5d7c072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 shutdown_connections 2023-10-19T10:38:04.826 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.821+0000 7f5d827d2640 1 -- 172.21.15.50:0/1517993143 wait complete. 2023-10-19T10:38:04.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:04.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.941+0000 7fa8f4beb640 1 Processor -- start 2023-10-19T10:38:04.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.941+0000 7fa8f4beb640 1 -- start start 2023-10-19T10:38:04.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8f4beb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa8f015a310 0x7fa8f006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8f4beb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8f4beb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00ccb00 con 0x7fa8f00ca690 2023-10-19T10:38:04.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8f4beb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f006d370 con 0x7fa8f015a310 2023-10-19T10:38:04.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8f4beb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f00ccde0 con 0x7fa8f006cf00 2023-10-19T10:38:04.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa8f015a310 0x7fa8f006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8eed9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa8f015a310 0x7fa8f006c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:51914/0 (socket says 172.21.15.50:51914) 2023-10-19T10:38:04.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/2783749259 learned_addr learned my addr 172.21.15.50:0/2783749259 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8ee59c640 1 --1- 172.21.15.50:0/2783749259 >> v1:172.21.15.183:6789/0 conn(0x7fa8f00ca690 0x7fa8f00cd8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:48602/0 (socket says 172.21.15.50:48602) 2023-10-19T10:38:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 msgr2=0x7fa8f00c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 --2- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/2783749259 >> v1:172.21.15.183:6789/0 conn(0x7fa8f00ca690 legacy=0x7fa8f00cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/2783749259 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00cd0c0 con 0x7fa8f015a310 2023-10-19T10:38:04.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.942+0000 7fa8eed9d640 1 --2- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:04.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.943+0000 7fa8e5d9b640 1 --2- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f006c9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009870 tx=0x7fa8d802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f4b608f9e3501f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.943+0000 7fa8edd9b640 1 -- 172.21.15.50:0/2783749259 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8030840 con 0x7fa8f015a310 2023-10-19T10:38:04.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.943+0000 7fa8edd9b640 1 -- 172.21.15.50:0/2783749259 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d80309a0 con 0x7fa8f015a310 2023-10-19T10:38:04.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.943+0000 7fa8edd9b640 1 -- 172.21.15.50:0/2783749259 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8030d30 con 0x7fa8f015a310 2023-10-19T10:38:04.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 -- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 msgr2=0x7fa8f006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f006c9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009870 tx=0x7fa8d802b980 comp rx=0 tx=0).stop 2023-10-19T10:38:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 -- 172.21.15.50:0/2783749259 shutdown_connections 2023-10-19T10:38:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/2783749259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f006c9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 -- 172.21.15.50:0/2783749259 >> 172.21.15.50:0/2783749259 conn(0x7fa8f00afbe0 msgr2=0x7fa8f00affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 -- 172.21.15.50:0/2783749259 shutdown_connections 2023-10-19T10:38:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.944+0000 7fa8f4beb640 1 -- 172.21.15.50:0/2783749259 wait complete. 2023-10-19T10:38:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.945+0000 7fa8f4beb640 1 Processor -- start 2023-10-19T10:38:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.945+0000 7fa8f4beb640 1 -- start start 2023-10-19T10:38:04.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8f4beb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8f4beb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 0x7fa8f0155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8f4beb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8f4beb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f00c4870 con 0x7fa8f00ca690 2023-10-19T10:38:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8f4beb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f014f750 con 0x7fa8f015a310 2023-10-19T10:38:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8f4beb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f014fa30 con 0x7fa8f006cf00 2023-10-19T10:38:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8e5d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 0x7fa8f0155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8e5d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 0x7fa8f0155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55708/0 (socket says 172.21.15.50:55708) 2023-10-19T10:38:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/1947470943 learned_addr learned my addr 172.21.15.50:0/1947470943 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8eed9d640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.946+0000 7fa8ee59c640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 msgr2=0x7fa8f00c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e5d9b640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 msgr2=0x7fa8f00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e5d9b640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e5d9b640 1 -- 172.21.15.50:0/1947470943 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8009a70 con 0x7fa8f00ca690 2023-10-19T10:38:04.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e5d9b640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 0x7fa8f0155a30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80090b0 tx=0x7fa8d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d803e040 con 0x7fa8f00ca690 2023-10-19T10:38:04.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8039cd0 con 0x7fa8f00ca690 2023-10-19T10:38:04.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d80094d0 con 0x7fa8f00ca690 2023-10-19T10:38:04.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8043680 con 0x7fa8f00ca690 2023-10-19T10:38:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.947+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f01567f0 con 0x7fa8f00ca690 2023-10-19T10:38:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.948+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f00c9830 con 0x7fa8f00ca690 2023-10-19T10:38:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.948+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8d80437e0 con 0x7fa8f00ca690 2023-10-19T10:38:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.949+0000 7fa8e77fe640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa8d006a050 0x7fa8d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.949+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa8d80aa870 con 0x7fa8f00ca690 2023-10-19T10:38:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.949+0000 7fa8ee59c640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa8d006a050 0x7fa8d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.950+0000 7fa8ee59c640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa8d006a050 0x7fa8d006c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0000f60 tx=0x7fa8e0007e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:04.953+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa8d8075cf0 con 0x7fa8f00ca690 2023-10-19T10:38:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.129+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8f00cd370 con 0x7fa8f00ca690 2023-10-19T10:38:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.129+0000 7fa8e77fe640 1 -- 172.21.15.50:0/1947470943 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa8d8079310 con 0x7fa8f00ca690 2023-10-19T10:38:05.132 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa8d006a050 msgr2=0x7fa8d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa8d006a050 0x7fa8d006c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0000f60 tx=0x7fa8e0007e20 comp rx=0 tx=0).stop 2023-10-19T10:38:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 msgr2=0x7fa8f0155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 0x7fa8f0155a30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80090b0 tx=0x7fa8d8004730 comp rx=0 tx=0).stop 2023-10-19T10:38:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 shutdown_connections 2023-10-19T10:38:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa8d006a050 0x7fa8d006c510 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f015a310 0x7fa8f00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8f00ca690 0x7fa8f0155a30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 --2- 172.21.15.50:0/1947470943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8f006cf00 0x7fa8f00c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 >> 172.21.15.50:0/1947470943 conn(0x7fa8f00afbe0 msgr2=0x7fa8f0158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 shutdown_connections 2023-10-19T10:38:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.131+0000 7fa8f4beb640 1 -- 172.21.15.50:0/1947470943 wait complete. 2023-10-19T10:38:05.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:38:05.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.249+0000 7fc1b5b26640 1 Processor -- start 2023-10-19T10:38:05.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.249+0000 7fc1b5b26640 1 -- start start 2023-10-19T10:38:05.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.249+0000 7fc1b5b26640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc1b00d2db0 0x7fc1b015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.249+0000 7fc1b5b26640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b006aa30 con 0x7fc1b015c570 2023-10-19T10:38:05.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.249+0000 7fc1b5b26640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b006b2d0 con 0x7fc1b00d0e30 2023-10-19T10:38:05.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.249+0000 7fc1b5b26640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b015ce90 con 0x7fc1b00d2db0 2023-10-19T10:38:05.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1af7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc1b015c570 0x7fc1b006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46864/0 (socket says 172.21.15.50:46864) 2023-10-19T10:38:05.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1af7fe640 1 -- 172.21.15.50:0/2562405068 learned_addr learned my addr 172.21.15.50:0/2562405068 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:05.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1affff640 1 --2- 172.21.15.50:0/2562405068 >> v2:172.21.15.183:3300/0 conn(0x7fc1b00d2db0 0x7fc1b015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149884097 0 0) 0x7fc1b006aa30 con 0x7fc1b015c570 2023-10-19T10:38:05.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc198003520 con 0x7fc1b015c570 2023-10-19T10:38:05.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527435160 0 0) 0x7fc198003520 con 0x7fc1b015c570 2023-10-19T10:38:05.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 >> v2:172.21.15.183:3300/0 conn(0x7fc1b00d2db0 msgr2=0x7fc1b015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:05.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 --2- 172.21.15.50:0/2562405068 >> v2:172.21.15.183:3300/0 conn(0x7fc1b00d2db0 0x7fc1b015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 >> v1:172.21.15.183:6789/0 conn(0x7fc1b00d0e30 legacy=0x7fc1b00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b00d1270 con 0x7fc1b015c570 2023-10-19T10:38:05.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fc1a00017f0 con 0x7fc1b015c570 2023-10-19T10:38:05.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1affff640 1 --2- 172.21.15.50:0/2562405068 >> v2:172.21.15.183:3300/0 conn(0x7fc1b00d2db0 0x7fc1b015c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:05.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 >> v1:172.21.15.50:6789/0 conn(0x7fc1b015c570 legacy=0x7fc1b006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc198003970 0x7fc198003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1980047c0 0x7fc198004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc198005180 0x7fc19800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b015ce90 con 0x7fc1980047c0 2023-10-19T10:38:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19800c120 con 0x7fc198003970 2023-10-19T10:38:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19800c2e0 con 0x7fc198005180 2023-10-19T10:38:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1980047c0 0x7fc198004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.250+0000 7fc1af7fe640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc198003970 0x7fc198003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1affff640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc198005180 0x7fc19800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1aeffd640 1 -- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc198005180 msgr2=0x7fc19800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc198005180 0x7fc19800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1aeffd640 1 -- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc198003970 msgr2=0x7fc198003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc198003970 0x7fc198003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1aeffd640 1 -- 172.21.15.50:0/2562405068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198003520 con 0x7fc1980047c0 2023-10-19T10:38:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1affff640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc198005180 0x7fc19800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1affff640 1 -- 172.21.15.50:0/2562405068 reap_dead start 2023-10-19T10:38:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.251+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1980047c0 0x7fc198004c40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4001750 tx=0x7fc1980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5c4f11c5257f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40052d0 con 0x7fc1980047c0 2023-10-19T10:38:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a402e050 con 0x7fc1980047c0 2023-10-19T10:38:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1ae7fc640 1 -- 172.21.15.50:0/2562405068 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4003830 con 0x7fc1980047c0 2023-10-19T10:38:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 -- 172.21.15.50:0/2562405068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1980047c0 msgr2=0x7fc198004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1980047c0 0x7fc198004c40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4001750 tx=0x7fc1980057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 -- 172.21.15.50:0/2562405068 shutdown_connections 2023-10-19T10:38:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/2562405068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1980047c0 0x7fc198004c40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 -- 172.21.15.50:0/2562405068 >> 172.21.15.50:0/2562405068 conn(0x7fc1b0089ff0 msgr2=0x7fc1b009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 -- 172.21.15.50:0/2562405068 shutdown_connections 2023-10-19T10:38:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.252+0000 7fc1b5b26640 1 -- 172.21.15.50:0/2562405068 wait complete. 2023-10-19T10:38:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.253+0000 7fc1b5b26640 1 Processor -- start 2023-10-19T10:38:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.253+0000 7fc1b5b26640 1 -- start start 2023-10-19T10:38:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1b5b26640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1980047c0 0x7fc1b015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1b5b26640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 0x7fc1b015f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1b5b26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1b00c7830 0x7fc1b015f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1b5b26640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c9170 con 0x7fc1b015cfd0 2023-10-19T10:38:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1b5b26640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c8910 con 0x7fc1b00c7830 2023-10-19T10:38:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1b5b26640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c8bb0 con 0x7fc1980047c0 2023-10-19T10:38:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 0x7fc1b015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 0x7fc1b015f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55734/0 (socket says 172.21.15.50:55734) 2023-10-19T10:38:05.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.254+0000 7fc1aeffd640 1 -- 172.21.15.50:0/3197850937 learned_addr learned my addr 172.21.15.50:0/3197850937 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:05.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1aeffd640 1 -- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1980047c0 msgr2=0x7fc1b015a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1affff640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1b00c7830 0x7fc1b015f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1980047c0 0x7fc1b015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1aeffd640 1 -- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1b00c7830 msgr2=0x7fc1b015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1b00c7830 0x7fc1b015f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1aeffd640 1 -- 172.21.15.50:0/3197850937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b00c8e90 con 0x7fc1b015cfd0 2023-10-19T10:38:05.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1aeffd640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 0x7fc1b015f070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc1a402bc00 tx=0x7fc1a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40076c0 con 0x7fc1b015cfd0 2023-10-19T10:38:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4007a30 con 0x7fc1b015cfd0 2023-10-19T10:38:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4006be0 con 0x7fc1b015cfd0 2023-10-19T10:38:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b00b53d0 con 0x7fc1b015cfd0 2023-10-19T10:38:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.255+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b015abe0 con 0x7fc1b015cfd0 2023-10-19T10:38:05.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.256+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b0073990 con 0x7fc1b015cfd0 2023-10-19T10:38:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.256+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc1a4006d40 con 0x7fc1b015cfd0 2023-10-19T10:38:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.257+0000 7fc197fff640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc19c06a050 0x7fc19c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.257+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc1a40aa4c0 con 0x7fc1b015cfd0 2023-10-19T10:38:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.257+0000 7fc1af7fe640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc19c06a050 0x7fc19c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.258+0000 7fc1af7fe640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc19c06a050 0x7fc19c06c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00011d0 tx=0x7fc1a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.261+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc1a40793a0 con 0x7fc1b015cfd0 2023-10-19T10:38:05.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.451+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc1b00d3480 con 0x7fc1b015cfd0 2023-10-19T10:38:05.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.451+0000 7fc197fff640 1 -- 172.21.15.50:0/3197850937 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc1a4078f30 con 0x7fc1b015cfd0 2023-10-19T10:38:05.454 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:38:05.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc19c06a050 msgr2=0x7fc19c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc19c06a050 0x7fc19c06c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00011d0 tx=0x7fc1a0009040 comp rx=0 tx=0).stop 2023-10-19T10:38:05.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 msgr2=0x7fc1b015f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 0x7fc1b015f070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc1a402bc00 tx=0x7fc1a4001120 comp rx=0 tx=0).stop 2023-10-19T10:38:05.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 shutdown_connections 2023-10-19T10:38:05.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc19c06a050 0x7fc19c06c510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1b00c7830 0x7fc1b015f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1b015cfd0 0x7fc1b015f070 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 --2- 172.21.15.50:0/3197850937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1980047c0 0x7fc1b015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 >> 172.21.15.50:0/3197850937 conn(0x7fc1b0089ff0 msgr2=0x7fc1b00cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 shutdown_connections 2023-10-19T10:38:05.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.453+0000 7fc1b5b26640 1 -- 172.21.15.50:0/3197850937 wait complete. 2023-10-19T10:38:05.467 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:38:05.467 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:05.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:05.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.626+0000 7f70a0102640 1 Processor -- start 2023-10-19T10:38:05.629 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.626+0000 7f70a0102640 1 -- start start 2023-10-19T10:38:05.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.627+0000 7f70a0102640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f70980cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.627+0000 7f70a0102640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7098160f30 con 0x7f709815fbc0 2023-10-19T10:38:05.631 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.627+0000 7f70a0102640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980d02c0 con 0x7f7098160aa0 2023-10-19T10:38:05.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.627+0000 7f70a0102640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980d05a0 con 0x7f70980b2400 2023-10-19T10:38:05.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.627+0000 7f709de9e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f709815fbc0 0x7f70980b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46866/0 (socket says 172.21.15.50:46866) 2023-10-19T10:38:05.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.627+0000 7f709de9e640 1 -- 172.21.15.50:0/52958446 learned_addr learned my addr 172.21.15.50:0/52958446 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:05.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709d69d640 1 --2- 172.21.15.50:0/52958446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f70980cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.632 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346136642 0 0) 0x7f7098160f30 con 0x7f709815fbc0 2023-10-19T10:38:05.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7080003520 con 0x7f709815fbc0 2023-10-19T10:38:05.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610719626 0 0) 0x7f70980d02c0 con 0x7f7098160aa0 2023-10-19T10:38:05.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7098160f30 con 0x7f7098160aa0 2023-10-19T10:38:05.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801703199 0 0) 0x7f7080003520 con 0x7f709815fbc0 2023-10-19T10:38:05.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709d69d640 1 -- 172.21.15.50:0/52958446 >> v1:172.21.15.183:6789/0 conn(0x7f7098160aa0 legacy=0x7f709815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709d69d640 1 -- 172.21.15.50:0/52958446 >> v1:172.21.15.50:6789/0 conn(0x7f709815fbc0 legacy=0x7f70980b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.628+0000 7f709d69d640 1 -- 172.21.15.50:0/52958446 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980d0880 con 0x7f70980b2400 2023-10-19T10:38:05.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f709d69d640 1 --2- 172.21.15.50:0/52958446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f70980cfc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f708c0097d0 tx=0x7f708c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9d5f99a36bec18 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090001690 con 0x7f70980b2400 2023-10-19T10:38:05.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c02e050 con 0x7f70980b2400 2023-10-19T10:38:05.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f709ce9c640 1 -- 172.21.15.50:0/52958446 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c030730 con 0x7f70980b2400 2023-10-19T10:38:05.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f70a0102640 1 -- 172.21.15.50:0/52958446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 msgr2=0x7f70980cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f70a0102640 1 --2- 172.21.15.50:0/52958446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f70980cfc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f708c0097d0 tx=0x7f708c02b800 comp rx=0 tx=0).stop 2023-10-19T10:38:05.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.629+0000 7f70a0102640 1 -- 172.21.15.50:0/52958446 shutdown_connections 2023-10-19T10:38:05.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.630+0000 7f70a0102640 1 --2- 172.21.15.50:0/52958446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f70980cfc00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.630+0000 7f70a0102640 1 -- 172.21.15.50:0/52958446 >> 172.21.15.50:0/52958446 conn(0x7f709806f3f0 msgr2=0x7f709806f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:05.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.630+0000 7f70a0102640 1 -- 172.21.15.50:0/52958446 shutdown_connections 2023-10-19T10:38:05.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.630+0000 7f70a0102640 1 -- 172.21.15.50:0/52958446 wait complete. 2023-10-19T10:38:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.631+0000 7f70a0102640 1 Processor -- start 2023-10-19T10:38:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.631+0000 7f70a0102640 1 -- start start 2023-10-19T10:38:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.631+0000 7f70a0102640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f7098150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.631+0000 7f70a0102640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709815fbc0 0x7f70980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70a0102640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 0x7f70980b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70a0102640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c9ce0 con 0x7f7098160aa0 2023-10-19T10:38:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70a0102640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70981511b0 con 0x7f709815fbc0 2023-10-19T10:38:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70a0102640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c8d80 con 0x7f70980b2400 2023-10-19T10:38:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 0x7f70980b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 0x7f70980b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55750/0 (socket says 172.21.15.50:55750) 2023-10-19T10:38:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 -- 172.21.15.50:0/2451497816 learned_addr learned my addr 172.21.15.50:0/2451497816 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709d69d640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709815fbc0 0x7f70980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709de9e640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f7098150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 -- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 msgr2=0x7f7098150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f7098150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 -- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709815fbc0 msgr2=0x7f70980b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709815fbc0 0x7f70980b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 -- 172.21.15.50:0/2451497816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c009400 con 0x7f7098160aa0 2023-10-19T10:38:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f709e69f640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 0x7f70980b52d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f708800c790 tx=0x7f708800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800cea0 con 0x7f7098160aa0 2023-10-19T10:38:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7088003b70 con 0x7f7098160aa0 2023-10-19T10:38:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70980c9060 con 0x7f7098160aa0 2023-10-19T10:38:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.632+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800a850 con 0x7f7098160aa0 2023-10-19T10:38:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.633+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980c9620 con 0x7f7098160aa0 2023-10-19T10:38:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.633+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7098073990 con 0x7f7098160aa0 2023-10-19T10:38:05.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.634+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f708800a9b0 con 0x7f7098160aa0 2023-10-19T10:38:05.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.634+0000 7f70967fc640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f708406a050 0x7f708406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.634+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7088088640 con 0x7f7098160aa0 2023-10-19T10:38:05.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.635+0000 7f709de9e640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f708406a050 0x7f708406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.635+0000 7f709de9e640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f708406a050 0x7f708406c510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7090001b70 tx=0x7f7090001b00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.639+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7088053ab0 con 0x7f7098160aa0 2023-10-19T10:38:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.816+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70980b61d0 con 0x7f7098160aa0 2023-10-19T10:38:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.817+0000 7f70967fc640 1 -- 172.21.15.50:0/2451497816 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f70880570d0 con 0x7f7098160aa0 2023-10-19T10:38:05.820 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:05.821 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f708406a050 msgr2=0x7f708406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f708406a050 0x7f708406c510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7090001b70 tx=0x7f7090001b00 comp rx=0 tx=0).stop 2023-10-19T10:38:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 msgr2=0x7f70980b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 0x7f70980b52d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f708800c790 tx=0x7f708800cc50 comp rx=0 tx=0).stop 2023-10-19T10:38:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 shutdown_connections 2023-10-19T10:38:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f708406a050 0x7f708406c510 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7098160aa0 0x7f70980b52d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709815fbc0 0x7f70980b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 --2- 172.21.15.50:0/2451497816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70980b2400 0x7f7098150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.819+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 >> 172.21.15.50:0/2451497816 conn(0x7f709806f3f0 msgr2=0x7f709806f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.820+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 shutdown_connections 2023-10-19T10:38:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.820+0000 7f70a0102640 1 -- 172.21.15.50:0/2451497816 wait complete. 2023-10-19T10:38:05.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:38:05.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.940+0000 7fda63157640 1 Processor -- start 2023-10-19T10:38:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.940+0000 7fda63157640 1 -- start start 2023-10-19T10:38:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda63157640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0b4070 0x7fda5c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda63157640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fda5c15cdc0 0x7fda5c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda63157640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0d3480 con 0x7fda5c15c960 2023-10-19T10:38:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda63157640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0b4990 con 0x7fda5c15cdc0 2023-10-19T10:38:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda63157640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15d280 con 0x7fda5c0b4070 2023-10-19T10:38:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fda5c15cdc0 0x7fda5c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fda5c15cdc0 0x7fda5c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55762/0 (socket says 172.21.15.50:55762) 2023-10-19T10:38:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 -- 172.21.15.50:0/1800949516 learned_addr learned my addr 172.21.15.50:0/1800949516 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda5bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fda5c15c960 0x7fda5c0d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46872/0 (socket says 172.21.15.50:46872) 2023-10-19T10:38:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda60ef3640 1 --2- 172.21.15.50:0/1800949516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0b4070 0x7fda5c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 -- 172.21.15.50:0/1800949516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0b4070 msgr2=0x7fda5c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 --2- 172.21.15.50:0/1800949516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0b4070 0x7fda5c0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 -- 172.21.15.50:0/1800949516 >> v1:172.21.15.50:6789/0 conn(0x7fda5c15c960 legacy=0x7fda5c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda616f4640 1 -- 172.21.15.50:0/1800949516 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c15d560 con 0x7fda5c15cdc0 2023-10-19T10:38:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.941+0000 7fda5b7fe640 1 -- 172.21.15.50:0/1800949516 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515697755 0 0) 0x7fda5c0d3480 con 0x7fda5c15c960 2023-10-19T10:38:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.942+0000 7fda616f4640 1 --2- 172.21.15.50:0/1800949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c0cd4b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fda44009870 tx=0x7fda4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbf858dfab59b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.942+0000 7fda5b7fe640 1 -- 172.21.15.50:0/1800949516 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44030840 con 0x7fda5c15cdc0 2023-10-19T10:38:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.942+0000 7fda5b7fe640 1 -- 172.21.15.50:0/1800949516 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda440309a0 con 0x7fda5c15cdc0 2023-10-19T10:38:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.942+0000 7fda5b7fe640 1 -- 172.21.15.50:0/1800949516 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44030d30 con 0x7fda5c15cdc0 2023-10-19T10:38:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 -- 172.21.15.50:0/1800949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 msgr2=0x7fda5c0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 --2- 172.21.15.50:0/1800949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c0cd4b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fda44009870 tx=0x7fda4402b980 comp rx=0 tx=0).stop 2023-10-19T10:38:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 -- 172.21.15.50:0/1800949516 shutdown_connections 2023-10-19T10:38:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 --2- 172.21.15.50:0/1800949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c0cd4b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 --2- 172.21.15.50:0/1800949516 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c0b4070 0x7fda5c0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 -- 172.21.15.50:0/1800949516 >> 172.21.15.50:0/1800949516 conn(0x7fda5c09b970 msgr2=0x7fda5c0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 -- 172.21.15.50:0/1800949516 shutdown_connections 2023-10-19T10:38:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.943+0000 7fda63157640 1 -- 172.21.15.50:0/1800949516 wait complete. 2023-10-19T10:38:05.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.944+0000 7fda63157640 1 Processor -- start 2023-10-19T10:38:05.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.944+0000 7fda63157640 1 -- start start 2023-10-19T10:38:05.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda63157640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c0b4070 0x7fda5c0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda63157640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c15c960 0x7fda5c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda63157640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda63157640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c8f60 con 0x7fda5c15cdc0 2023-10-19T10:38:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda63157640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0b1480 con 0x7fda5c0b4070 2023-10-19T10:38:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda63157640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0b0870 con 0x7fda5c15c960 2023-10-19T10:38:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55766/0 (socket says 172.21.15.50:55766) 2023-10-19T10:38:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 -- 172.21.15.50:0/3684426192 learned_addr learned my addr 172.21.15.50:0/3684426192 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda60ef3640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c0b4070 0x7fda5c0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 -- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c15c960 msgr2=0x7fda5c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c15c960 0x7fda5c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 -- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c0b4070 msgr2=0x7fda5c0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c0b4070 0x7fda5c0b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 -- 172.21.15.50:0/3684426192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440094d0 con 0x7fda5c15cdc0 2023-10-19T10:38:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.945+0000 7fda616f4640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c150870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fda44009870 tx=0x7fda44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.946+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4403e040 con 0x7fda5c15cdc0 2023-10-19T10:38:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.946+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44039c10 con 0x7fda5c15cdc0 2023-10-19T10:38:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.946+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0b0b50 con 0x7fda5c15cdc0 2023-10-19T10:38:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.946+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44004030 con 0x7fda5c15cdc0 2023-10-19T10:38:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.946+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0b1110 con 0x7fda5c15cdc0 2023-10-19T10:38:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.947+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c0b1630 con 0x7fda5c15cdc0 2023-10-19T10:38:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.947+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fda44004190 con 0x7fda5c15cdc0 2023-10-19T10:38:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.947+0000 7fda597fa640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a050 0x7fda3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.948+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda440aaff0 con 0x7fda5c15cdc0 2023-10-19T10:38:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.948+0000 7fda60ef3640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a050 0x7fda3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.948+0000 7fda60ef3640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a050 0x7fda3c06c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0c69a0 tx=0x7fda5400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:05.952+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fda5c0b1630 con 0x7fda5c15cdc0 2023-10-19T10:38:06.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.128+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda5c065860 con 0x7fda5c15cdc0 2023-10-19T10:38:06.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.128+0000 7fda597fa640 1 -- 172.21.15.50:0/3684426192 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda44079f60 con 0x7fda5c15cdc0 2023-10-19T10:38:06.132 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:06.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a050 msgr2=0x7fda3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a050 0x7fda3c06c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0c69a0 tx=0x7fda5400b040 comp rx=0 tx=0).stop 2023-10-19T10:38:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 msgr2=0x7fda5c150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c150870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fda44009870 tx=0x7fda44004730 comp rx=0 tx=0).stop 2023-10-19T10:38:06.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 shutdown_connections 2023-10-19T10:38:06.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fda3c06a050 0x7fda3c06c510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda5c15cdc0 0x7fda5c150870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fda5c15c960 0x7fda5c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 --2- 172.21.15.50:0/3684426192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda5c0b4070 0x7fda5c0b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.135 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 >> 172.21.15.50:0/3684426192 conn(0x7fda5c09b970 msgr2=0x7fda5c06a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:06.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 shutdown_connections 2023-10-19T10:38:06.136 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.131+0000 7fda63157640 1 -- 172.21.15.50:0/3684426192 wait complete. 2023-10-19T10:38:06.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:06.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 Processor -- start 2023-10-19T10:38:06.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 -- start start 2023-10-19T10:38:06.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8c280ca690 0x7f8c280cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c2806d0d0 0x7f8c280c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280cde00 con 0x7f8c2815a310 2023-10-19T10:38:06.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280ccb00 con 0x7f8c2806d0d0 2023-10-19T10:38:06.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.251+0000 7f8c2dd1f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280ccde0 con 0x7f8c280ca690 2023-10-19T10:38:06.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c2806d0d0 0x7f8c280c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8c2806d0d0 0x7f8c280c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55782/0 (socket says 172.21.15.50:55782) 2023-10-19T10:38:06.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 -- 172.21.15.50:0/1454775522 learned_addr learned my addr 172.21.15.50:0/1454775522 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:06.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c277fe640 1 --2- 172.21.15.50:0/1454775522 >> v2:172.21.15.183:3300/0 conn(0x7f8c280ca690 0x7f8c280cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 -- 172.21.15.50:0/1454775522 >> v2:172.21.15.183:3300/0 conn(0x7f8c280ca690 msgr2=0x7f8c280cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:06.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 --2- 172.21.15.50:0/1454775522 >> v2:172.21.15.183:3300/0 conn(0x7f8c280ca690 0x7f8c280cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 -- 172.21.15.50:0/1454775522 >> v1:172.21.15.183:6789/0 conn(0x7f8c2815a310 legacy=0x7f8c2806c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.252+0000 7f8c27fff640 1 -- 172.21.15.50:0/1454775522 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280cd0c0 con 0x7f8c2806d0d0 2023-10-19T10:38:06.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c27fff640 1 --2- 172.21.15.50:0/1454775522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c280c8d30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c280cbc70 tx=0x7f8c2002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ac80ab57b7f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c267fc640 1 -- 172.21.15.50:0/1454775522 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2002cb20 con 0x7f8c2806d0d0 2023-10-19T10:38:06.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c267fc640 1 -- 172.21.15.50:0/1454775522 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2002cc80 con 0x7f8c2806d0d0 2023-10-19T10:38:06.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c267fc640 1 -- 172.21.15.50:0/1454775522 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200343f0 con 0x7f8c2806d0d0 2023-10-19T10:38:06.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/1454775522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 msgr2=0x7f8c280c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/1454775522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c280c8d30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c280cbc70 tx=0x7f8c2002bf80 comp rx=0 tx=0).stop 2023-10-19T10:38:06.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/1454775522 shutdown_connections 2023-10-19T10:38:06.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/1454775522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c280c8d30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/1454775522 >> v2:172.21.15.183:3300/0 conn(0x7f8c280ca690 0x7f8c280cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/1454775522 >> 172.21.15.50:0/1454775522 conn(0x7f8c280afbe0 msgr2=0x7f8c280affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:06.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.253+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/1454775522 shutdown_connections 2023-10-19T10:38:06.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.254+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/1454775522 wait complete. 2023-10-19T10:38:06.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 Processor -- start 2023-10-19T10:38:06.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 -- start start 2023-10-19T10:38:06.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c2815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c280ca690 0x7f8c28135a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c2815a310 0x7f8c2815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815dbe0 con 0x7f8c2806d0d0 2023-10-19T10:38:06.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815d900 con 0x7f8c280ca690 2023-10-19T10:38:06.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.255+0000 7f8c2dd1f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280c21a0 con 0x7f8c2815a310 2023-10-19T10:38:06.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c2815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c2815ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55792/0 (socket says 172.21.15.50:55792) 2023-10-19T10:38:06.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 -- 172.21.15.50:0/476332818 learned_addr learned my addr 172.21.15.50:0/476332818 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:06.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c26ffd640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c280ca690 0x7f8c28135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c27fff640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c2815a310 0x7f8c2815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 -- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c2815a310 msgr2=0x7f8c2815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c2815a310 0x7f8c2815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 -- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c280ca690 msgr2=0x7f8c28135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c280ca690 0x7f8c28135a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 -- 172.21.15.50:0/476332818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20009b30 con 0x7f8c2806d0d0 2023-10-19T10:38:06.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c26ffd640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c280ca690 0x7f8c28135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:06.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c277fe640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c2815ad80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e9b0 tx=0x7f8c1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18009670 con 0x7f8c2806d0d0 2023-10-19T10:38:06.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1800c9a0 con 0x7f8c2806d0d0 2023-10-19T10:38:06.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c280c2480 con 0x7f8c2806d0d0 2023-10-19T10:38:06.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800cd50 con 0x7f8c2806d0d0 2023-10-19T10:38:06.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.256+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2815d3d0 con 0x7f8c2806d0d0 2023-10-19T10:38:06.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.257+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2804e420 con 0x7f8c2806d0d0 2023-10-19T10:38:06.269 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.258+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8c18013470 con 0x7f8c2806d0d0 2023-10-19T10:38:06.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.258+0000 7f8c0ffff640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c1406a050 0x7f8c1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.258+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8c18088b80 con 0x7f8c2806d0d0 2023-10-19T10:38:06.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.259+0000 7f8c26ffd640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c1406a050 0x7f8c1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.259+0000 7f8c26ffd640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c1406a050 0x7f8c1406c510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c280c9240 tx=0x7f8c1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.262+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f8c18053f30 con 0x7f8c2806d0d0 2023-10-19T10:38:06.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.439+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8c280cc760 con 0x7f8c2806d0d0 2023-10-19T10:38:06.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.439+0000 7f8c0ffff640 1 -- 172.21.15.50:0/476332818 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c18057550 con 0x7f8c2806d0d0 2023-10-19T10:38:06.442 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:06.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c1406a050 msgr2=0x7f8c1406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c1406a050 0x7f8c1406c510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c280c9240 tx=0x7f8c1c009040 comp rx=0 tx=0).stop 2023-10-19T10:38:06.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 msgr2=0x7f8c2815ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c2815ad80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e9b0 tx=0x7f8c1800ee70 comp rx=0 tx=0).stop 2023-10-19T10:38:06.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 shutdown_connections 2023-10-19T10:38:06.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8c1406a050 0x7f8c1406c510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c2815a310 0x7f8c2815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c280ca690 0x7f8c28135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 --2- 172.21.15.50:0/476332818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2806d0d0 0x7f8c2815ad80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 >> 172.21.15.50:0/476332818 conn(0x7f8c280afbe0 msgr2=0x7f8c281573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:06.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 shutdown_connections 2023-10-19T10:38:06.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.441+0000 7f8c2dd1f640 1 -- 172.21.15.50:0/476332818 wait complete. 2023-10-19T10:38:06.454 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:38:06.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 Processor -- start 2023-10-19T10:38:06.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 -- start start 2023-10-19T10:38:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe7140d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe7140b3090 0x7fe714160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140af250 con 0x7fe7140b3cd0 2023-10-19T10:38:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140b0190 con 0x7fe7140b3090 2023-10-19T10:38:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.562+0000 7fe719e8b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140af510 con 0x7fe7140b38f0 2023-10-19T10:38:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe7140d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe712ffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe7140b3090 0x7fe714160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe7140d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:47840/0 (socket says 172.21.15.50:47840) 2023-10-19T10:38:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 -- 172.21.15.50:0/772765320 learned_addr learned my addr 172.21.15.50:0/772765320 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe713fff640 1 --1- 172.21.15.50:0/772765320 >> v1:172.21.15.183:6789/0 conn(0x7fe7140b3cd0 0x7fe7140b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:48644/0 (socket says 172.21.15.50:48644) 2023-10-19T10:38:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 -- 172.21.15.50:0/772765320 >> v2:172.21.15.183:3300/0 conn(0x7fe7140b3090 msgr2=0x7fe714160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 --2- 172.21.15.50:0/772765320 >> v2:172.21.15.183:3300/0 conn(0x7fe7140b3090 0x7fe714160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 -- 172.21.15.50:0/772765320 >> v1:172.21.15.183:6789/0 conn(0x7fe7140b3cd0 legacy=0x7fe7140b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe7137fe640 1 -- 172.21.15.50:0/772765320 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140af7f0 con 0x7fe7140b38f0 2023-10-19T10:38:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.563+0000 7fe712ffd640 1 --2- 172.21.15.50:0/772765320 >> v2:172.21.15.183:3300/0 conn(0x7fe7140b3090 0x7fe714160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.564+0000 7fe7137fe640 1 --2- 172.21.15.50:0/772765320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe7140d30e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe704009850 tx=0x7fe70402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403041883f7cd1c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.564+0000 7fe7127fc640 1 -- 172.21.15.50:0/772765320 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704004230 con 0x7fe7140b38f0 2023-10-19T10:38:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.564+0000 7fe7127fc640 1 -- 172.21.15.50:0/772765320 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704004390 con 0x7fe7140b38f0 2023-10-19T10:38:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.564+0000 7fe7127fc640 1 -- 172.21.15.50:0/772765320 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704003810 con 0x7fe7140b38f0 2023-10-19T10:38:06.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 -- 172.21.15.50:0/772765320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 msgr2=0x7fe7140d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 --2- 172.21.15.50:0/772765320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe7140d30e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe704009850 tx=0x7fe70402b960 comp rx=0 tx=0).stop 2023-10-19T10:38:06.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 -- 172.21.15.50:0/772765320 shutdown_connections 2023-10-19T10:38:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 --2- 172.21.15.50:0/772765320 >> v2:172.21.15.183:3300/0 conn(0x7fe7140b3090 0x7fe714160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 --2- 172.21.15.50:0/772765320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe7140d30e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 -- 172.21.15.50:0/772765320 >> 172.21.15.50:0/772765320 conn(0x7fe71409b960 msgr2=0x7fe7140d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 -- 172.21.15.50:0/772765320 shutdown_connections 2023-10-19T10:38:06.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.565+0000 7fe719e8b640 1 -- 172.21.15.50:0/772765320 wait complete. 2023-10-19T10:38:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.566+0000 7fe719e8b640 1 Processor -- start 2023-10-19T10:38:06.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.566+0000 7fe719e8b640 1 -- start start 2023-10-19T10:38:06.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe719e8b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 0x7fe71414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe719e8b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe71415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe719e8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7140b3cd0 0x7fe71415f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe719e8b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714156dc0 con 0x7fe7140b3090 2023-10-19T10:38:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe719e8b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141570a0 con 0x7fe7140b3cd0 2023-10-19T10:38:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe719e8b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe714157380 con 0x7fe7140b38f0 2023-10-19T10:38:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 0x7fe71414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 0x7fe71414acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55806/0 (socket says 172.21.15.50:55806) 2023-10-19T10:38:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 -- 172.21.15.50:0/141983080 learned_addr learned my addr 172.21.15.50:0/141983080 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe712ffd640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe71415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe713fff640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7140b3cd0 0x7fe71415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 -- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 msgr2=0x7fe71415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe71415f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 -- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7140b3cd0 msgr2=0x7fe71415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7140b3cd0 0x7fe71415f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.567+0000 7fe7137fe640 1 -- 172.21.15.50:0/141983080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708009400 con 0x7fe7140b3090 2023-10-19T10:38:06.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.568+0000 7fe7137fe640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 0x7fe71414acd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe70402b9a0 tx=0x7fe7040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.568+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70403e070 con 0x7fe7140b3090 2023-10-19T10:38:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.568+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704004500 con 0x7fe7140b3090 2023-10-19T10:38:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.568+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe704009460 con 0x7fe7140b3090 2023-10-19T10:38:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.568+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704003df0 con 0x7fe7140b3090 2023-10-19T10:38:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.568+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71414b210 con 0x7fe7140b3090 2023-10-19T10:38:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.569+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71404e420 con 0x7fe7140b3090 2023-10-19T10:38:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.569+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe704006040 con 0x7fe7140b3090 2023-10-19T10:38:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.570+0000 7fe6fbfff640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe70006a050 0x7fe70006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.570+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe7040aa640 con 0x7fe7140b3090 2023-10-19T10:38:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.570+0000 7fe712ffd640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe70006a050 0x7fe70006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.570+0000 7fe712ffd640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe70006a050 0x7fe70006c510 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe714157b00 tx=0x7fe708009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.574+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe704075a20 con 0x7fe7140b3090 2023-10-19T10:38:06.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.753+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe714156f00 con 0x7fe7140b3090 2023-10-19T10:38:06.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.753+0000 7fe6fbfff640 1 -- 172.21.15.50:0/141983080 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe714156f00 con 0x7fe7140b3090 2023-10-19T10:38:06.756 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe70006a050 msgr2=0x7fe70006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe70006a050 0x7fe70006c510 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe714157b00 tx=0x7fe708009190 comp rx=0 tx=0).stop 2023-10-19T10:38:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 msgr2=0x7fe71414acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 0x7fe71414acd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe70402b9a0 tx=0x7fe7040046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 shutdown_connections 2023-10-19T10:38:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe70006a050 0x7fe70006c510 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7140b3cd0 0x7fe71415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7140b38f0 0x7fe71415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 --2- 172.21.15.50:0/141983080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7140b3090 0x7fe71414acd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 >> 172.21.15.50:0/141983080 conn(0x7fe71409b960 msgr2=0x7fe7140ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:06.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 shutdown_connections 2023-10-19T10:38:06.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.755+0000 7fe719e8b640 1 -- 172.21.15.50:0/141983080 wait complete. 2023-10-19T10:38:06.768 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:38:06.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.875+0000 7faed6451640 1 Processor -- start 2023-10-19T10:38:06.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.875+0000 7faed6451640 1 -- start start 2023-10-19T10:38:06.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed6451640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faed00b1f60 0x7faed00cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed6451640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faed00b3330 0x7faed00b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed6451640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00caaa0 con 0x7faed006fcb0 2023-10-19T10:38:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed6451640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed0070140 con 0x7faed00b3330 2023-10-19T10:38:06.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed6451640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00cf760 con 0x7faed00b1f60 2023-10-19T10:38:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faed00b3330 0x7faed00b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faed00b3330 0x7faed00b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49744/0 (socket says 172.21.15.50:49744) 2023-10-19T10:38:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faecf7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7faed006fcb0 0x7faed00b2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39236/0 (socket says 172.21.15.50:39236) 2023-10-19T10:38:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 -- 172.21.15.50:0/3266867128 learned_addr learned my addr 172.21.15.50:0/3266867128 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:06.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faecffff640 1 --2- 172.21.15.50:0/3266867128 >> v2:172.21.15.183:3300/0 conn(0x7faed00b1f60 0x7faed00cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 -- 172.21.15.50:0/3266867128 >> v2:172.21.15.183:3300/0 conn(0x7faed00b1f60 msgr2=0x7faed00cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 --2- 172.21.15.50:0/3266867128 >> v2:172.21.15.183:3300/0 conn(0x7faed00b1f60 0x7faed00cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 -- 172.21.15.50:0/3266867128 >> v1:172.21.15.50:6789/0 conn(0x7faed006fcb0 legacy=0x7faed00b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faeceffd640 1 -- 172.21.15.50:0/3266867128 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020254344 0 0) 0x7faed00caaa0 con 0x7faed006fcb0 2023-10-19T10:38:06.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.876+0000 7faed49ee640 1 -- 172.21.15.50:0/3266867128 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00cfbb0 con 0x7faed00b3330 2023-10-19T10:38:06.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.877+0000 7faed49ee640 1 --2- 172.21.15.50:0/3266867128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3330 0x7faed00b37a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faed0072c90 tx=0x7faec802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed05f7da9198618 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.877+0000 7faeceffd640 1 -- 172.21.15.50:0/3266867128 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec802ca80 con 0x7faed00b3330 2023-10-19T10:38:06.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.877+0000 7faeceffd640 1 -- 172.21.15.50:0/3266867128 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec802cbe0 con 0x7faed00b3330 2023-10-19T10:38:06.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.877+0000 7faeceffd640 1 -- 172.21.15.50:0/3266867128 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec80343f0 con 0x7faed00b3330 2023-10-19T10:38:06.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 -- 172.21.15.50:0/3266867128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3330 msgr2=0x7faed00b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 --2- 172.21.15.50:0/3266867128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3330 0x7faed00b37a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faed0072c90 tx=0x7faec802bee0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 -- 172.21.15.50:0/3266867128 shutdown_connections 2023-10-19T10:38:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 --2- 172.21.15.50:0/3266867128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b3330 0x7faed00b37a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 --2- 172.21.15.50:0/3266867128 >> v2:172.21.15.183:3300/0 conn(0x7faed00b1f60 0x7faed00cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 -- 172.21.15.50:0/3266867128 >> 172.21.15.50:0/3266867128 conn(0x7faed00ccf60 msgr2=0x7faed00cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 -- 172.21.15.50:0/3266867128 shutdown_connections 2023-10-19T10:38:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.878+0000 7faed6451640 1 -- 172.21.15.50:0/3266867128 wait complete. 2023-10-19T10:38:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.879+0000 7faed6451640 1 Processor -- start 2023-10-19T10:38:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.879+0000 7faed6451640 1 -- start start 2023-10-19T10:38:06.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faed6451640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed006fcb0 0x7faed015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faed6451640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 0x7faed00b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faed6451640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed00b3330 0x7faed00c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faed6451640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00ca1c0 con 0x7faed00b1f60 2023-10-19T10:38:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faed6451640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed01f2cc0 con 0x7faed00b3330 2023-10-19T10:38:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faed6451640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed01f2f60 con 0x7faed006fcb0 2023-10-19T10:38:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 0x7faed00b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 0x7faed00b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49750/0 (socket says 172.21.15.50:49750) 2023-10-19T10:38:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 -- 172.21.15.50:0/1618212670 learned_addr learned my addr 172.21.15.50:0/1618212670 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:06.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 -- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed006fcb0 msgr2=0x7faed015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed006fcb0 0x7faed015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 -- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed00b3330 msgr2=0x7faed00c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:06.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed00b3330 0x7faed00c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 -- 172.21.15.50:0/1618212670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8009b30 con 0x7faed00b1f60 2023-10-19T10:38:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.880+0000 7faecf7fe640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 0x7faed00b1900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faec000c430 tx=0x7faec000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.881+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00074e0 con 0x7faed00b1f60 2023-10-19T10:38:06.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.881+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec0007660 con 0x7faed00b1f60 2023-10-19T10:38:06.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.881+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0004a20 con 0x7faed00b1f60 2023-10-19T10:38:06.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.881+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed01f3240 con 0x7faed00b1f60 2023-10-19T10:38:06.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.881+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01f3800 con 0x7faed00b1f60 2023-10-19T10:38:06.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.882+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed004e420 con 0x7faed00b1f60 2023-10-19T10:38:06.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.882+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faec0004b80 con 0x7faed00b1f60 2023-10-19T10:38:06.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.882+0000 7faeccff9640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a050 0x7faebc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:06.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.883+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faec0088750 con 0x7faed00b1f60 2023-10-19T10:38:06.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.883+0000 7faecffff640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a050 0x7faebc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:06.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.883+0000 7faecffff640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a050 0x7faebc06c510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faec4006fd0 tx=0x7faec4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:06.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:06.887+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7faec0053b10 con 0x7faed00b1f60 2023-10-19T10:38:07.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.061+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faed00cc1b0 con 0x7faed00b1f60 2023-10-19T10:38:07.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.061+0000 7faeccff9640 1 -- 172.21.15.50:0/1618212670 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7faec0057130 con 0x7faed00b1f60 2023-10-19T10:38:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:38:07.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a050 msgr2=0x7faebc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a050 0x7faebc06c510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faec4006fd0 tx=0x7faec4008040 comp rx=0 tx=0).stop 2023-10-19T10:38:07.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 msgr2=0x7faed00b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 0x7faed00b1900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faec000c430 tx=0x7faec000c8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 shutdown_connections 2023-10-19T10:38:07.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faebc06a050 0x7faebc06c510 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faed00b3330 0x7faed00c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faed00b1f60 0x7faed00b1900 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 --2- 172.21.15.50:0/1618212670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faed006fcb0 0x7faed015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 >> 172.21.15.50:0/1618212670 conn(0x7faed00ccf60 msgr2=0x7faed00cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:07.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 shutdown_connections 2023-10-19T10:38:07.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.063+0000 7faed6451640 1 -- 172.21.15.50:0/1618212670 wait complete. 2023-10-19T10:38:07.076 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:07.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 Processor -- start 2023-10-19T10:38:07.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 -- start start 2023-10-19T10:38:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b38f0 0x7f0a3c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0a3c0b3090 0x7f0a3c160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0a3c0b3cd0 0x7f0a3c161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0b0190 con 0x7f0a3c0b3cd0 2023-10-19T10:38:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0af250 con 0x7f0a3c0b3090 2023-10-19T10:38:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.185+0000 7f0a42e73640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0af510 con 0x7f0a3c0b38f0 2023-10-19T10:38:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0a3c0b3cd0 0x7f0a3c161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0a3c0b3cd0 0x7f0a3c161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49764/0 (socket says 172.21.15.50:49764) 2023-10-19T10:38:07.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 -- 172.21.15.50:0/2618523560 learned_addr learned my addr 172.21.15.50:0/2618523560 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a3bfff640 1 --2- 172.21.15.50:0/2618523560 >> v2:172.21.15.183:3300/0 conn(0x7f0a3c0b3090 0x7f0a3c160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a40c0f640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b38f0 0x7f0a3c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 -- 172.21.15.50:0/2618523560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b38f0 msgr2=0x7f0a3c0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b38f0 0x7f0a3c0d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 -- 172.21.15.50:0/2618523560 >> v2:172.21.15.183:3300/0 conn(0x7f0a3c0b3090 msgr2=0x7f0a3c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 --2- 172.21.15.50:0/2618523560 >> v2:172.21.15.183:3300/0 conn(0x7f0a3c0b3090 0x7f0a3c160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a41410640 1 -- 172.21.15.50:0/2618523560 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0af7f0 con 0x7f0a3c0b3cd0 2023-10-19T10:38:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.186+0000 7f0a40c0f640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b38f0 0x7f0a3c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a41410640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3cd0 0x7f0a3c161040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c08b140 tx=0x7f0a3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b730537baeb35ef1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a3b7fe640 1 -- 172.21.15.50:0/2618523560 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3402ca70 con 0x7f0a3c0b3cd0 2023-10-19T10:38:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a3b7fe640 1 -- 172.21.15.50:0/2618523560 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3402cbd0 con 0x7f0a3c0b3cd0 2023-10-19T10:38:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a3b7fe640 1 -- 172.21.15.50:0/2618523560 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340343f0 con 0x7f0a3c0b3cd0 2023-10-19T10:38:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 -- 172.21.15.50:0/2618523560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3cd0 msgr2=0x7f0a3c161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3cd0 0x7f0a3c161040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c08b140 tx=0x7f0a3402bed0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 -- 172.21.15.50:0/2618523560 shutdown_connections 2023-10-19T10:38:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3cd0 0x7f0a3c161040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2618523560 >> v2:172.21.15.183:3300/0 conn(0x7f0a3c0b3090 0x7f0a3c160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2618523560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b38f0 0x7f0a3c0d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 -- 172.21.15.50:0/2618523560 >> 172.21.15.50:0/2618523560 conn(0x7f0a3c09b960 msgr2=0x7f0a3c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:07.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 -- 172.21.15.50:0/2618523560 shutdown_connections 2023-10-19T10:38:07.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.187+0000 7f0a42e73640 1 -- 172.21.15.50:0/2618523560 wait complete. 2023-10-19T10:38:07.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.188+0000 7f0a42e73640 1 Processor -- start 2023-10-19T10:38:07.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 -- start start 2023-10-19T10:38:07.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 0x7f0a3c154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a3c0b38f0 0x7f0a3c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b3cd0 0x7f0a3c15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c15ac80 con 0x7f0a3c0b3090 2023-10-19T10:38:07.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c155d80 con 0x7f0a3c0b38f0 2023-10-19T10:38:07.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a42e73640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c1574d0 con 0x7f0a3c0b3cd0 2023-10-19T10:38:07.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.189+0000 7f0a40c0f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 0x7f0a3c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 0x7f0a3c154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49776/0 (socket says 172.21.15.50:49776) 2023-10-19T10:38:07.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 -- 172.21.15.50:0/2066457919 learned_addr learned my addr 172.21.15.50:0/2066457919 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:07.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a3bfff640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a3c0b38f0 0x7f0a3c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 -- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b3cd0 msgr2=0x7f0a3c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a41410640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b3cd0 0x7f0a3c15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b3cd0 0x7f0a3c15df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 -- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a3c0b38f0 msgr2=0x7f0a3c15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a3c0b38f0 0x7f0a3c15b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 -- 172.21.15.50:0/2066457919 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34009b30 con 0x7f0a3c0b3090 2023-10-19T10:38:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a40c0f640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 0x7f0a3c154ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400e810 tx=0x7f0a2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24004030 con 0x7f0a3c0b3090 2023-10-19T10:38:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a24004190 con 0x7f0a3c0b3090 2023-10-19T10:38:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c1577b0 con 0x7f0a3c0b3090 2023-10-19T10:38:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24005990 con 0x7f0a3c0b3090 2023-10-19T10:38:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.190+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c15a8a0 con 0x7f0a3c0b3090 2023-10-19T10:38:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.191+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c15aef0 con 0x7f0a3c0b3090 2023-10-19T10:38:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.191+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a24005af0 con 0x7f0a3c0b3090 2023-10-19T10:38:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.192+0000 7f0a397fa640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0a1c06a050 0x7f0a1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.192+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0a24088850 con 0x7f0a3c0b3090 2023-10-19T10:38:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.192+0000 7f0a3bfff640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0a1c06a050 0x7f0a1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.193+0000 7f0a3bfff640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0a1c06a050 0x7f0a1c06c510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c15b940 tx=0x7f0a2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.196+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0a24053c10 con 0x7f0a3c0b3090 2023-10-19T10:38:07.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.370+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0a3c04e420 con 0x7f0a3c0b3090 2023-10-19T10:38:07.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.371+0000 7f0a397fa640 1 -- 172.21.15.50:0/2066457919 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a24057230 con 0x7f0a3c0b3090 2023-10-19T10:38:07.374 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0a1c06a050 msgr2=0x7f0a1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0a1c06a050 0x7f0a1c06c510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c15b940 tx=0x7f0a2c008040 comp rx=0 tx=0).stop 2023-10-19T10:38:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 msgr2=0x7f0a3c154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 0x7f0a3c154ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400e810 tx=0x7f0a2400ecd0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 shutdown_connections 2023-10-19T10:38:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0a1c06a050 0x7f0a1c06c510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0a3c0b3cd0 0x7f0a3c15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a3c0b38f0 0x7f0a3c15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 --2- 172.21.15.50:0/2066457919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0b3090 0x7f0a3c154ee0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.372+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 >> 172.21.15.50:0/2066457919 conn(0x7f0a3c09b960 msgr2=0x7f0a3c0ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.373+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 shutdown_connections 2023-10-19T10:38:07.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.373+0000 7f0a42e73640 1 -- 172.21.15.50:0/2066457919 wait complete. 2023-10-19T10:38:07.386 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:38:07.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.491+0000 7fc285702640 1 Processor -- start 2023-10-19T10:38:07.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.491+0000 7fc285702640 1 -- start start 2023-10-19T10:38:07.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.491+0000 7fc285702640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc2800ce3c0 0x7fc28015d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.491+0000 7fc285702640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2801610a0 con 0x7fc2800b1cf0 2023-10-19T10:38:07.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.491+0000 7fc285702640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2800ce7a0 con 0x7fc280160810 2023-10-19T10:38:07.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.491+0000 7fc285702640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28015cad0 con 0x7fc2800ce3c0 2023-10-19T10:38:07.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc27effd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc2800ce3c0 0x7fc28015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc27effd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc2800ce3c0 0x7fc28015d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49786/0 (socket says 172.21.15.50:49786) 2023-10-19T10:38:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc27effd640 1 -- 172.21.15.50:0/2650544675 learned_addr learned my addr 172.21.15.50:0/2650544675 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc277fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc2800b1cf0 0x7fc280160100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39250/0 (socket says 172.21.15.50:39250) 2023-10-19T10:38:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc27effd640 1 -- 172.21.15.50:0/2650544675 >> v1:172.21.15.183:6789/0 conn(0x7fc280160810 legacy=0x7fc2800cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc27effd640 1 -- 172.21.15.50:0/2650544675 >> v1:172.21.15.50:6789/0 conn(0x7fc2800b1cf0 legacy=0x7fc280160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.492+0000 7fc27effd640 1 -- 172.21.15.50:0/2650544675 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280160dc0 con 0x7fc2800ce3c0 2023-10-19T10:38:07.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.493+0000 7fc27effd640 1 --2- 172.21.15.50:0/2650544675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2800ce3c0 0x7fc28015d790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc270009870 tx=0x7fc27002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cd30b39b3daa56 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.493+0000 7fc27e7fc640 1 -- 172.21.15.50:0/2650544675 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27002f840 con 0x7fc2800ce3c0 2023-10-19T10:38:07.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.493+0000 7fc27e7fc640 1 -- 172.21.15.50:0/2650544675 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27002f9a0 con 0x7fc2800ce3c0 2023-10-19T10:38:07.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.493+0000 7fc27e7fc640 1 -- 172.21.15.50:0/2650544675 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27002fd30 con 0x7fc2800ce3c0 2023-10-19T10:38:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 -- 172.21.15.50:0/2650544675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2800ce3c0 msgr2=0x7fc28015d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 --2- 172.21.15.50:0/2650544675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2800ce3c0 0x7fc28015d790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc270009870 tx=0x7fc27002ba20 comp rx=0 tx=0).stop 2023-10-19T10:38:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 -- 172.21.15.50:0/2650544675 shutdown_connections 2023-10-19T10:38:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 --2- 172.21.15.50:0/2650544675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2800ce3c0 0x7fc28015d790 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 -- 172.21.15.50:0/2650544675 >> 172.21.15.50:0/2650544675 conn(0x7fc28006f3f0 msgr2=0x7fc28006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 -- 172.21.15.50:0/2650544675 shutdown_connections 2023-10-19T10:38:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.494+0000 7fc285702640 1 -- 172.21.15.50:0/2650544675 wait complete. 2023-10-19T10:38:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.495+0000 7fc285702640 1 Processor -- start 2023-10-19T10:38:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.495+0000 7fc285702640 1 -- start start 2023-10-19T10:38:07.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc285702640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2800b1cf0 0x7fc28015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc285702640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2800ce3c0 0x7fc28015f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc285702640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 0x7fc2800c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc285702640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800ca300 con 0x7fc280160810 2023-10-19T10:38:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc285702640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800c7840 con 0x7fc2800b1cf0 2023-10-19T10:38:07.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc285702640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800c7a90 con 0x7fc2800ce3c0 2023-10-19T10:38:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 0x7fc2800c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 0x7fc2800c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49788/0 (socket says 172.21.15.50:49788) 2023-10-19T10:38:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 -- 172.21.15.50:0/1791100345 learned_addr learned my addr 172.21.15.50:0/1791100345 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27effd640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2800b1cf0 0x7fc28015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc277fff640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2800ce3c0 0x7fc28015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 -- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2800ce3c0 msgr2=0x7fc28015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2800ce3c0 0x7fc28015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 -- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2800b1cf0 msgr2=0x7fc28015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2800b1cf0 0x7fc28015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 -- 172.21.15.50:0/1791100345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2700094d0 con 0x7fc280160810 2023-10-19T10:38:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 0x7fc2800c6120 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc27800c6e0 tx=0x7fc27800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.496+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278003b70 con 0x7fc280160810 2023-10-19T10:38:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.497+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc278003cf0 con 0x7fc280160810 2023-10-19T10:38:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.497+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278005310 con 0x7fc280160810 2023-10-19T10:38:07.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.497+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2800c7d70 con 0x7fc280160810 2023-10-19T10:38:07.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.497+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2800c8330 con 0x7fc280160810 2023-10-19T10:38:07.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.498+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2800d10d0 con 0x7fc280160810 2023-10-19T10:38:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.498+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc278005470 con 0x7fc280160810 2023-10-19T10:38:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.498+0000 7fc2777fe640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26006a050 0x7fc26006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.498+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2780884e0 con 0x7fc280160810 2023-10-19T10:38:07.508 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.498+0000 7fc27effd640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26006a050 0x7fc26006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.499+0000 7fc27effd640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26006a050 0x7fc26006c510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc270036000 tx=0x7fc270008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.503+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc278053950 con 0x7fc280160810 2023-10-19T10:38:07.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.691+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc280073990 con 0x7fc280160810 2023-10-19T10:38:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.691+0000 7fc2777fe640 1 -- 172.21.15.50:0/1791100345 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc278003e60 con 0x7fc280160810 2023-10-19T10:38:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:38:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26006a050 msgr2=0x7fc26006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26006a050 0x7fc26006c510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc270036000 tx=0x7fc270008030 comp rx=0 tx=0).stop 2023-10-19T10:38:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 msgr2=0x7fc2800c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 0x7fc2800c6120 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc27800c6e0 tx=0x7fc27800cba0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 shutdown_connections 2023-10-19T10:38:07.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26006a050 0x7fc26006c510 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc280160810 0x7fc2800c6120 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2800ce3c0 0x7fc28015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 --2- 172.21.15.50:0/1791100345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2800b1cf0 0x7fc28015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 >> 172.21.15.50:0/1791100345 conn(0x7fc28006f3f0 msgr2=0x7fc2800cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:07.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 shutdown_connections 2023-10-19T10:38:07.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.693+0000 7fc285702640 1 -- 172.21.15.50:0/1791100345 wait complete. 2023-10-19T10:38:07.706 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-19T10:38:07.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:07.707 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-19T10:38:07.737 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:07.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:38:07.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:07.739 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-19T10:38:07.793 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:38:07.794 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:07.812 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:38:07.813 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:07.871 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:07.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.974+0000 7fb216770640 1 Processor -- start 2023-10-19T10:38:07.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.974+0000 7fb216770640 1 -- start start 2023-10-19T10:38:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb216770640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb216770640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb210070e90 0x7fb2100cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb216770640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100718a0 con 0x7fb2100ccd10 2023-10-19T10:38:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb216770640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21006e610 con 0x7fb210070e90 2023-10-19T10:38:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb216770640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100ca8d0 con 0x7fb210159550 2023-10-19T10:38:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb20f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb2100ccd10 0x7fb210070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39264/0 (socket says 172.21.15.50:39264) 2023-10-19T10:38:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.975+0000 7fb20f7fe640 1 -- 172.21.15.50:0/1482273019 learned_addr learned my addr 172.21.15.50:0/1482273019 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb214d0d640 1 --2- 172.21.15.50:0/1482273019 >> v2:172.21.15.183:3300/0 conn(0x7fb210070e90 0x7fb2100cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb20ffff640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb20effd640 1 -- 172.21.15.50:0/1482273019 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169457068 0 0) 0x7fb2100718a0 con 0x7fb2100ccd10 2023-10-19T10:38:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb20effd640 1 -- 172.21.15.50:0/1482273019 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f8003520 con 0x7fb2100ccd10 2023-10-19T10:38:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb214d0d640 1 -- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 msgr2=0x7fb2100b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:07.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb214d0d640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb214d0d640 1 -- 172.21.15.50:0/1482273019 >> v1:172.21.15.50:6789/0 conn(0x7fb2100ccd10 legacy=0x7fb210070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb214d0d640 1 -- 172.21.15.50:0/1482273019 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21006e860 con 0x7fb210070e90 2023-10-19T10:38:07.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.976+0000 7fb20ffff640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:07.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb214d0d640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb210070e90 0x7fb2100cad60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb208009d90 tx=0x7fb20802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc66950a73ee42fa server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb20effd640 1 -- 172.21.15.50:0/1482273019 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20802cc60 con 0x7fb210070e90 2023-10-19T10:38:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb20effd640 1 -- 172.21.15.50:0/1482273019 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20802cdc0 con 0x7fb210070e90 2023-10-19T10:38:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb20effd640 1 -- 172.21.15.50:0/1482273019 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208034640 con 0x7fb210070e90 2023-10-19T10:38:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 -- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb210070e90 msgr2=0x7fb2100cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb210070e90 0x7fb2100cad60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb208009d90 tx=0x7fb20802beb0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 -- 172.21.15.50:0/1482273019 shutdown_connections 2023-10-19T10:38:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb210070e90 0x7fb2100cad60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 --2- 172.21.15.50:0/1482273019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 -- 172.21.15.50:0/1482273019 >> 172.21.15.50:0/1482273019 conn(0x7fb2100cbbc0 msgr2=0x7fb2100cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:07.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 -- 172.21.15.50:0/1482273019 shutdown_connections 2023-10-19T10:38:07.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.977+0000 7fb216770640 1 -- 172.21.15.50:0/1482273019 wait complete. 2023-10-19T10:38:07.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 Processor -- start 2023-10-19T10:38:07.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 -- start start 2023-10-19T10:38:07.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 0x7fb2100c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2100ccd10 0x7fb2100c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100b4d70 con 0x7fb210070e90 2023-10-19T10:38:07.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100c1ee0 con 0x7fb2100ccd10 2023-10-19T10:38:07.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.979+0000 7fb216770640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100c21c0 con 0x7fb210159550 2023-10-19T10:38:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 0x7fb2100c09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 0x7fb2100c09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49804/0 (socket says 172.21.15.50:49804) 2023-10-19T10:38:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 -- 172.21.15.50:0/3272818978 learned_addr learned my addr 172.21.15.50:0/3272818978 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20f7fe640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2100ccd10 0x7fb2100c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb214d0d640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 -- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 msgr2=0x7fb2100c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 -- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2100ccd10 msgr2=0x7fb2100c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2100ccd10 0x7fb2100c0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:07.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 -- 172.21.15.50:0/3272818978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200009400 con 0x7fb210070e90 2023-10-19T10:38:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20ffff640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 0x7fb2100c09c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb20000e960 tx=0x7fb20000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000ca00 con 0x7fb210070e90 2023-10-19T10:38:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20000cb60 con 0x7fb210070e90 2023-10-19T10:38:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200013470 con 0x7fb210070e90 2023-10-19T10:38:07.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb208009a70 con 0x7fb210070e90 2023-10-19T10:38:07.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.980+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2100c1b90 con 0x7fb210070e90 2023-10-19T10:38:07.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.981+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb210073990 con 0x7fb210070e90 2023-10-19T10:38:07.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.982+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb2000135d0 con 0x7fb210070e90 2023-10-19T10:38:07.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.982+0000 7fb20cff9640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1fc06a080 0x7fb1fc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.982+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb200088a40 con 0x7fb210070e90 2023-10-19T10:38:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.982+0000 7fb20f7fe640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1fc06a080 0x7fb1fc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.983+0000 7fb20f7fe640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1fc06a080 0x7fb1fc06c540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb21015cbd0 tx=0x7fb204009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:07.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:07.986+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb20008ae60 con 0x7fb210070e90 2023-10-19T10:38:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.162+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2100ce950 con 0x7fb210070e90 2023-10-19T10:38:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.163+0000 7fb20cff9640 1 -- 172.21.15.50:0/3272818978 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fb200053eb0 con 0x7fb210070e90 2023-10-19T10:38:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:08.167 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:08.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1fc06a080 msgr2=0x7fb1fc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1fc06a080 0x7fb1fc06c540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb21015cbd0 tx=0x7fb204009040 comp rx=0 tx=0).stop 2023-10-19T10:38:08.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 msgr2=0x7fb2100c09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 0x7fb2100c09c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb20000e960 tx=0x7fb20000ee20 comp rx=0 tx=0).stop 2023-10-19T10:38:08.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 shutdown_connections 2023-10-19T10:38:08.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1fc06a080 0x7fb1fc06c540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb210159550 0x7fb2100c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2100ccd10 0x7fb2100c0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 --2- 172.21.15.50:0/3272818978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb210070e90 0x7fb2100c09c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 >> 172.21.15.50:0/3272818978 conn(0x7fb2100cbbc0 msgr2=0x7fb2100cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 shutdown_connections 2023-10-19T10:38:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.165+0000 7fb216770640 1 -- 172.21.15.50:0/3272818978 wait complete. 2023-10-19T10:38:08.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:38:08.180 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:08.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.289+0000 7f77e1507640 1 Processor -- start 2023-10-19T10:38:08.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.289+0000 7f77e1507640 1 -- start start 2023-10-19T10:38:08.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.289+0000 7f77e1507640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 0x7f77dc0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.289+0000 7f77e1507640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc160580 con 0x7f77dc15cc80 2023-10-19T10:38:08.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.289+0000 7f77e1507640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc160860 con 0x7f77dc06ef00 2023-10-19T10:38:08.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.289+0000 7f77e1507640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc160b40 con 0x7f77dc069800 2023-10-19T10:38:08.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77db7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f77dc15cc80 0x7f77dc06dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39270/0 (socket says 172.21.15.50:39270) 2023-10-19T10:38:08.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77db7fe640 1 -- 172.21.15.50:0/3196851408 learned_addr learned my addr 172.21.15.50:0/3196851408 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669942096 0 0) 0x7f77dc160580 con 0x7f77dc15cc80 2023-10-19T10:38:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77daffd640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 0x7f77dc0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c8003520 con 0x7f77dc15cc80 2023-10-19T10:38:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224541894 0 0) 0x7f77c8003520 con 0x7f77dc15cc80 2023-10-19T10:38:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 msgr2=0x7f77dc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 0x7f77dc0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 >> v1:172.21.15.183:6789/0 conn(0x7f77dc06ef00 legacy=0x7f77dc069be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc160e20 con 0x7f77dc15cc80 2023-10-19T10:38:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f77d40019f0 con 0x7f77dc15cc80 2023-10-19T10:38:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 >> v1:172.21.15.50:6789/0 conn(0x7f77dc15cc80 legacy=0x7f77dc06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c80039a0 0x7f77c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 0x7f77c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77c8004d00 0x7f77c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc160b40 con 0x7f77c8004d00 2023-10-19T10:38:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c800ba30 con 0x7f77c80039a0 2023-10-19T10:38:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c800bbf0 con 0x7f77dc069800 2023-10-19T10:38:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77db7fe640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77c8004d00 0x7f77c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.290+0000 7f77daffd640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c80039a0 0x7f77c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77d27fc640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 0x7f77c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77db7fe640 1 -- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 msgr2=0x7f77c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77db7fe640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc069800 0x7f77c80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77db7fe640 1 -- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c80039a0 msgr2=0x7f77c8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77db7fe640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c80039a0 0x7f77c8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77db7fe640 1 -- 172.21.15.50:0/3196851408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8003520 con 0x7f77c8004d00 2023-10-19T10:38:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77db7fe640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77c8004d00 0x7f77c8005180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f77d400a800 tx=0x7f77c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2442249bd3dacff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.291+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40072d0 con 0x7f77c8004d00 2023-10-19T10:38:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d402f070 con 0x7f77c8004d00 2023-10-19T10:38:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77da7fc640 1 -- 172.21.15.50:0/3196851408 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40076b0 con 0x7f77c8004d00 2023-10-19T10:38:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77e1507640 1 -- 172.21.15.50:0/3196851408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77c8004d00 msgr2=0x7f77c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77e1507640 1 --2- 172.21.15.50:0/3196851408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77c8004d00 0x7f77c8005180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f77d400a800 tx=0x7f77c80057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77db7fe640 1 -- 172.21.15.50:0/3196851408 reap_dead start 2023-10-19T10:38:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77e1507640 1 -- 172.21.15.50:0/3196851408 shutdown_connections 2023-10-19T10:38:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77e1507640 1 -- 172.21.15.50:0/3196851408 >> 172.21.15.50:0/3196851408 conn(0x7f77dc0cf170 msgr2=0x7f77dc0cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77e1507640 1 -- 172.21.15.50:0/3196851408 shutdown_connections 2023-10-19T10:38:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.292+0000 7f77e1507640 1 -- 172.21.15.50:0/3196851408 wait complete. 2023-10-19T10:38:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.293+0000 7f77e1507640 1 Processor -- start 2023-10-19T10:38:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.293+0000 7f77e1507640 1 -- start start 2023-10-19T10:38:08.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77e1507640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c8004d00 0x7f77dc0c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77e1507640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 0x7f77dc0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77e1507640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc0acfa0 0x7f77dc144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77e1507640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc1450d0 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77e1507640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0ad420 con 0x7f77c8004d00 2023-10-19T10:38:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77e1507640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c7f20 con 0x7f77dc0acfa0 2023-10-19T10:38:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77d27fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 0x7f77dc0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77d27fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 0x7f77dc0aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49824/0 (socket says 172.21.15.50:49824) 2023-10-19T10:38:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.294+0000 7f77d27fc640 1 -- 172.21.15.50:0/4047297282 learned_addr learned my addr 172.21.15.50:0/4047297282 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77daffd640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c8004d00 0x7f77dc0c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77db7fe640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc0acfa0 0x7f77dc144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d27fc640 1 -- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc0acfa0 msgr2=0x7f77dc144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d27fc640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc0acfa0 0x7f77dc144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d27fc640 1 -- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c8004d00 msgr2=0x7f77dc0c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d27fc640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c8004d00 0x7f77dc0c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d27fc640 1 -- 172.21.15.50:0/4047297282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc144df0 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77db7fe640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc0acfa0 0x7f77dc144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d27fc640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 0x7f77dc0aca60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f77c400e580 tx=0x7f77c400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40052e0 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4005440 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc0c81c0 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4003810 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.295+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc0c8780 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.296+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc04e420 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.297+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77c4003970 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.297+0000 7f77d3fff640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f77bc06a080 0x7f77bc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.297+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f77c4088690 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.297+0000 7f77daffd640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f77bc06a080 0x7f77bc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.298+0000 7f77daffd640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f77bc06a080 0x7f77bc06c540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009b00 tx=0x7f77cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.301+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f77c40574f0 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.476+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77dc06a640 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.477+0000 7f77d3fff640 1 -- 172.21.15.50:0/4047297282 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77c4057080 con 0x7f77dc0c8ca0 2023-10-19T10:38:08.482 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:08.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.478+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f77bc06a080 msgr2=0x7f77bc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.478+0000 7f77e1507640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f77bc06a080 0x7f77bc06c540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009b00 tx=0x7f77cc008040 comp rx=0 tx=0).stop 2023-10-19T10:38:08.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 msgr2=0x7f77dc0aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 0x7f77dc0aca60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f77c400e580 tx=0x7f77c400ea40 comp rx=0 tx=0).stop 2023-10-19T10:38:08.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 shutdown_connections 2023-10-19T10:38:08.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f77bc06a080 0x7f77bc06c540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77dc0acfa0 0x7f77dc144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77dc0c8ca0 0x7f77dc0aca60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 --2- 172.21.15.50:0/4047297282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c8004d00 0x7f77dc0c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 >> 172.21.15.50:0/4047297282 conn(0x7f77dc0cf170 msgr2=0x7f77dc15e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:08.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 shutdown_connections 2023-10-19T10:38:08.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.479+0000 7f77e1507640 1 -- 172.21.15.50:0/4047297282 wait complete. 2023-10-19T10:38:08.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:08.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.599+0000 7fb845cc4640 1 Processor -- start 2023-10-19T10:38:08.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.599+0000 7fb845cc4640 1 -- start start 2023-10-19T10:38:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb845cc4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 0x7fb8400d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb845cc4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 0x7fb8400cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb845cc4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84015d280 con 0x7fb8400b4070 2023-10-19T10:38:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb845cc4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84015d560 con 0x7fb84015c960 2023-10-19T10:38:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb845cc4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015d840 con 0x7fb84015cdf0 2023-10-19T10:38:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 0x7fb8400cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb83effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 0x7fb8400d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 0x7fb8400cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:48602/0 (socket says 172.21.15.50:48602) 2023-10-19T10:38:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb83effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 0x7fb8400d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:58118/0 (socket says 172.21.15.50:58118) 2023-10-19T10:38:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb83ffff640 1 -- 172.21.15.50:0/467541437 learned_addr learned my addr 172.21.15.50:0/467541437 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.600+0000 7fb83f7fe640 1 --1- 172.21.15.50:0/467541437 >> v1:172.21.15.183:6789/0 conn(0x7fb8400b4070 0x7fb8400b4450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:37614/0 (socket says 172.21.15.50:37614) 2023-10-19T10:38:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.601+0000 7fb83ffff640 1 -- 172.21.15.50:0/467541437 >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 msgr2=0x7fb8400d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.601+0000 7fb83ffff640 1 --2- 172.21.15.50:0/467541437 >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 0x7fb8400d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.601+0000 7fb83ffff640 1 -- 172.21.15.50:0/467541437 >> v1:172.21.15.183:6789/0 conn(0x7fb8400b4070 legacy=0x7fb8400b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.601+0000 7fb83ffff640 1 -- 172.21.15.50:0/467541437 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84015db20 con 0x7fb84015cdf0 2023-10-19T10:38:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.601+0000 7fb83effd640 1 --2- 172.21.15.50:0/467541437 >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 0x7fb8400d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:08.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb83ffff640 1 --2- 172.21.15.50:0/467541437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 0x7fb8400cd4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb838009e50 tx=0x7fb83802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1160de03179c0958 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb83e7fc640 1 -- 172.21.15.50:0/467541437 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83802cb00 con 0x7fb84015cdf0 2023-10-19T10:38:08.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb83e7fc640 1 -- 172.21.15.50:0/467541437 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83802cc60 con 0x7fb84015cdf0 2023-10-19T10:38:08.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb83e7fc640 1 -- 172.21.15.50:0/467541437 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380343f0 con 0x7fb84015cdf0 2023-10-19T10:38:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 -- 172.21.15.50:0/467541437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 msgr2=0x7fb8400cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 --2- 172.21.15.50:0/467541437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 0x7fb8400cd4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb838009e50 tx=0x7fb83802bf60 comp rx=0 tx=0).stop 2023-10-19T10:38:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 -- 172.21.15.50:0/467541437 shutdown_connections 2023-10-19T10:38:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 --2- 172.21.15.50:0/467541437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb84015cdf0 0x7fb8400cd4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 --2- 172.21.15.50:0/467541437 >> v2:172.21.15.183:3300/0 conn(0x7fb84015c960 0x7fb8400d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 -- 172.21.15.50:0/467541437 >> 172.21.15.50:0/467541437 conn(0x7fb84009b970 msgr2=0x7fb8400d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 -- 172.21.15.50:0/467541437 shutdown_connections 2023-10-19T10:38:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.602+0000 7fb845cc4640 1 -- 172.21.15.50:0/467541437 wait complete. 2023-10-19T10:38:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.603+0000 7fb845cc4640 1 Processor -- start 2023-10-19T10:38:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 -- start start 2023-10-19T10:38:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8400b4070 0x7fb840159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 0x7fb8400b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8401f1f00 0x7fb8400ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb840159f50 con 0x7fb84015c960 2023-10-19T10:38:08.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015a230 con 0x7fb8401f1f00 2023-10-19T10:38:08.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb845cc4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015a510 con 0x7fb8400b4070 2023-10-19T10:38:08.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb83effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 0x7fb8400b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb83effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 0x7fb8400b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49842/0 (socket says 172.21.15.50:49842) 2023-10-19T10:38:08.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.604+0000 7fb83effd640 1 -- 172.21.15.50:0/2085852208 learned_addr learned my addr 172.21.15.50:0/2085852208 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:08.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83ffff640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8401f1f00 0x7fb8400ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83f7fe640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8400b4070 0x7fb840159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83effd640 1 -- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8400b4070 msgr2=0x7fb840159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83effd640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8400b4070 0x7fb840159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83effd640 1 -- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8401f1f00 msgr2=0x7fb8400ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83effd640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8401f1f00 0x7fb8400ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83effd640 1 -- 172.21.15.50:0/2085852208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834009400 con 0x7fb84015c960 2023-10-19T10:38:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb83effd640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 0x7fb8400b1090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb83400ebe0 tx=0x7fb83400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834009760 con 0x7fb84015c960 2023-10-19T10:38:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400cc30 con 0x7fb84015c960 2023-10-19T10:38:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb838009b30 con 0x7fb84015c960 2023-10-19T10:38:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400eee0 con 0x7fb84015c960 2023-10-19T10:38:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.605+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84015aad0 con 0x7fb84015c960 2023-10-19T10:38:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.606+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84004e420 con 0x7fb84015c960 2023-10-19T10:38:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.607+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb834013470 con 0x7fb84015c960 2023-10-19T10:38:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.607+0000 7fb827fff640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb82c06a050 0x7fb82c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.607+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb834088b70 con 0x7fb84015c960 2023-10-19T10:38:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.607+0000 7fb83f7fe640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb82c06a050 0x7fb82c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.608+0000 7fb83f7fe640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb82c06a050 0x7fb82c06c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb830000fb0 tx=0x7fb830007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.612+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb834057a50 con 0x7fb84015c960 2023-10-19T10:38:08.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.791+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb84008c720 con 0x7fb84015c960 2023-10-19T10:38:08.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.791+0000 7fb827fff640 1 -- 172.21.15.50:0/2085852208 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb8340575e0 con 0x7fb84015c960 2023-10-19T10:38:08.795 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:08.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb82c06a050 msgr2=0x7fb82c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb82c06a050 0x7fb82c06c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb830000fb0 tx=0x7fb830007c90 comp rx=0 tx=0).stop 2023-10-19T10:38:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 msgr2=0x7fb8400b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 0x7fb8400b1090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb83400ebe0 tx=0x7fb83400c520 comp rx=0 tx=0).stop 2023-10-19T10:38:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 shutdown_connections 2023-10-19T10:38:08.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb82c06a050 0x7fb82c06c510 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8401f1f00 0x7fb8400ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb84015c960 0x7fb8400b1090 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 --2- 172.21.15.50:0/2085852208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb8400b4070 0x7fb840159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 >> 172.21.15.50:0/2085852208 conn(0x7fb84009b970 msgr2=0x7fb84015dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 shutdown_connections 2023-10-19T10:38:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.793+0000 7fb845cc4640 1 -- 172.21.15.50:0/2085852208 wait complete. 2023-10-19T10:38:08.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:38:08.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.914+0000 7f0ee5217640 1 Processor -- start 2023-10-19T10:38:08.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.914+0000 7f0ee5217640 1 -- start start 2023-10-19T10:38:08.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0ee5217640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0ee00b21d0 0x7f0ee00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0ee5217640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee00b25b0 0x7f0ee00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0ee5217640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee00cfc90 con 0x7f0ee006f540 2023-10-19T10:38:08.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0ee5217640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee015efb0 con 0x7f0ee00b21d0 2023-10-19T10:38:08.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0ee5217640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee015e3a0 con 0x7f0ee00b25b0 2023-10-19T10:38:08.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0ee00b21d0 0x7f0ee00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0ee00b21d0 0x7f0ee00ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49844/0 (socket says 172.21.15.50:49844) 2023-10-19T10:38:08.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 -- 172.21.15.50:0/2012791363 learned_addr learned my addr 172.21.15.50:0/2012791363 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:08.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0ede59c640 1 --2- 172.21.15.50:0/2012791363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee00b25b0 0x7f0ee00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 -- 172.21.15.50:0/2012791363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee00b25b0 msgr2=0x7f0ee00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:08.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 --2- 172.21.15.50:0/2012791363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee00b25b0 0x7f0ee00cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 -- 172.21.15.50:0/2012791363 >> v1:172.21.15.183:6789/0 conn(0x7f0ee006f540 legacy=0x7f0ee006f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.915+0000 7f0eded9d640 1 -- 172.21.15.50:0/2012791363 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee015e680 con 0x7f0ee00b21d0 2023-10-19T10:38:08.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.916+0000 7f0eded9d640 1 --2- 172.21.15.50:0/2012791363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b21d0 0x7f0ee00ce020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009870 tx=0x7f0ed802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7712009ba18581d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.916+0000 7f0eddd9b640 1 -- 172.21.15.50:0/2012791363 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8030840 con 0x7f0ee00b21d0 2023-10-19T10:38:08.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.916+0000 7f0eddd9b640 1 -- 172.21.15.50:0/2012791363 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed80309a0 con 0x7f0ee00b21d0 2023-10-19T10:38:08.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.916+0000 7f0eddd9b640 1 -- 172.21.15.50:0/2012791363 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8030d30 con 0x7f0ee00b21d0 2023-10-19T10:38:08.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 -- 172.21.15.50:0/2012791363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b21d0 msgr2=0x7f0ee00ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 --2- 172.21.15.50:0/2012791363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b21d0 0x7f0ee00ce020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009870 tx=0x7f0ed802b980 comp rx=0 tx=0).stop 2023-10-19T10:38:08.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 -- 172.21.15.50:0/2012791363 shutdown_connections 2023-10-19T10:38:08.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 --2- 172.21.15.50:0/2012791363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee00b25b0 0x7f0ee00cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 --2- 172.21.15.50:0/2012791363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b21d0 0x7f0ee00ce020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 -- 172.21.15.50:0/2012791363 >> 172.21.15.50:0/2012791363 conn(0x7f0ee00ccf60 msgr2=0x7f0ee00cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:08.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 -- 172.21.15.50:0/2012791363 shutdown_connections 2023-10-19T10:38:08.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.917+0000 7f0ee5217640 1 -- 172.21.15.50:0/2012791363 wait complete. 2023-10-19T10:38:08.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.918+0000 7f0ee5217640 1 Processor -- start 2023-10-19T10:38:08.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.918+0000 7f0ee5217640 1 -- start start 2023-10-19T10:38:08.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ee5217640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee006f540 0x7f0ee0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ee5217640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee00b21d0 0x7f0ee01f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ee5217640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 0x7f0ee00c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ee5217640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00c5ff0 con 0x7f0ee00b25b0 2023-10-19T10:38:08.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ee5217640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00c6270 con 0x7f0ee00b21d0 2023-10-19T10:38:08.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ee5217640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee01f3da0 con 0x7f0ee006f540 2023-10-19T10:38:08.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0edf59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 0x7f0ee00c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0edf59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 0x7f0ee00c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49848/0 (socket says 172.21.15.50:49848) 2023-10-19T10:38:08.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0edf59e640 1 -- 172.21.15.50:0/84474572 learned_addr learned my addr 172.21.15.50:0/84474572 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:08.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0eded9d640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee006f540 0x7f0ee0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.919+0000 7f0ede59c640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee00b21d0 0x7f0ee01f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0edf59e640 1 -- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee006f540 msgr2=0x7f0ee0157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0edf59e640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee006f540 0x7f0ee0157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0edf59e640 1 -- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee00b21d0 msgr2=0x7f0ee01f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:08.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0edf59e640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee00b21d0 0x7f0ee01f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:08.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0edf59e640 1 -- 172.21.15.50:0/84474572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80094d0 con 0x7f0ee00b25b0 2023-10-19T10:38:08.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0edf59e640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 0x7f0ee00c6a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c6b0 tx=0x7f0ed000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003b70 con 0x7f0ee00b25b0 2023-10-19T10:38:08.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0003cf0 con 0x7f0ee00b25b0 2023-10-19T10:38:08.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee01f4080 con 0x7f0ee00b25b0 2023-10-19T10:38:08.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003ea0 con 0x7f0ee00b25b0 2023-10-19T10:38:08.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.920+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee00af250 con 0x7f0ee00b25b0 2023-10-19T10:38:08.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.921+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee00af3e0 con 0x7f0ee00b25b0 2023-10-19T10:38:08.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.922+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ed0005310 con 0x7f0ee00b25b0 2023-10-19T10:38:08.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.922+0000 7f0ed77fe640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ec006a050 0x7f0ec006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:08.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.922+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0ed0088490 con 0x7f0ee00b25b0 2023-10-19T10:38:08.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.922+0000 7f0eded9d640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ec006a050 0x7f0ec006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:08.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.923+0000 7f0eded9d640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ec006a050 0x7f0ec006c510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8036000 tx=0x7f0ed8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:08.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:08.927+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0ed0053990 con 0x7f0ee00b25b0 2023-10-19T10:38:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.102+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ee004e420 con 0x7f0ee00b25b0 2023-10-19T10:38:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.103+0000 7f0ed77fe640 1 -- 172.21.15.50:0/84474572 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ed0056fb0 con 0x7f0ee00b25b0 2023-10-19T10:38:09.106 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:09.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ec006a050 msgr2=0x7f0ec006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ec006a050 0x7f0ec006c510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8036000 tx=0x7f0ed8008030 comp rx=0 tx=0).stop 2023-10-19T10:38:09.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 msgr2=0x7f0ee00c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 0x7f0ee00c6a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c6b0 tx=0x7f0ed000cb70 comp rx=0 tx=0).stop 2023-10-19T10:38:09.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 shutdown_connections 2023-10-19T10:38:09.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ec006a050 0x7f0ec006c510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee00b25b0 0x7f0ee00c6a50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee00b21d0 0x7f0ee01f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 --2- 172.21.15.50:0/84474572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee006f540 0x7f0ee0157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 >> 172.21.15.50:0/84474572 conn(0x7f0ee00ccf60 msgr2=0x7f0ee015af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.104+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 shutdown_connections 2023-10-19T10:38:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.105+0000 7f0ee5217640 1 -- 172.21.15.50:0/84474572 wait complete. 2023-10-19T10:38:09.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:38:09.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.224+0000 7f1b37560640 1 Processor -- start 2023-10-19T10:38:09.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.225+0000 7f1b37560640 1 -- start start 2023-10-19T10:38:09.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.225+0000 7f1b37560640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b300b3090 0x7f1b300d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.225+0000 7f1b37560640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b300af250 con 0x7f1b30161210 2023-10-19T10:38:09.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.225+0000 7f1b37560640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b300af510 con 0x7f1b300b38f0 2023-10-19T10:38:09.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.225+0000 7f1b37560640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b300b0190 con 0x7f1b300b3090 2023-10-19T10:38:09.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b34afb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b300b3090 0x7f1b300d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b35afd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1b30161210 0x7f1b300b05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39284/0 (socket says 172.21.15.50:39284) 2023-10-19T10:38:09.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b35afd640 1 -- 172.21.15.50:0/1769360194 learned_addr learned my addr 172.21.15.50:0/1769360194 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:09.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b34afb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b300b3090 0x7f1b300d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49864/0 (socket says 172.21.15.50:49864) 2023-10-19T10:38:09.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b34afb640 1 -- 172.21.15.50:0/1769360194 >> v1:172.21.15.183:6789/0 conn(0x7f1b300b38f0 legacy=0x7f1b30160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:09.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b34afb640 1 -- 172.21.15.50:0/1769360194 >> v1:172.21.15.50:6789/0 conn(0x7f1b30161210 legacy=0x7f1b300b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b34afb640 1 -- 172.21.15.50:0/1769360194 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300af7f0 con 0x7f1b300b3090 2023-10-19T10:38:09.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.226+0000 7f1b2ffff640 1 -- 172.21.15.50:0/1769360194 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194374503 0 0) 0x7f1b300af250 con 0x7f1b30161210 2023-10-19T10:38:09.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b34afb640 1 --2- 172.21.15.50:0/1769360194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b300b3090 0x7f1b300d30e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b240097a0 tx=0x7f1b2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396be98051aefed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b2ffff640 1 -- 172.21.15.50:0/1769360194 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24031830 con 0x7f1b300b3090 2023-10-19T10:38:09.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b2ffff640 1 -- 172.21.15.50:0/1769360194 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b24031990 con 0x7f1b300b3090 2023-10-19T10:38:09.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b2ffff640 1 -- 172.21.15.50:0/1769360194 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24031d20 con 0x7f1b300b3090 2023-10-19T10:38:09.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 -- 172.21.15.50:0/1769360194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b300b3090 msgr2=0x7f1b300d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 --2- 172.21.15.50:0/1769360194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b300b3090 0x7f1b300d30e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b240097a0 tx=0x7f1b2402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 -- 172.21.15.50:0/1769360194 shutdown_connections 2023-10-19T10:38:09.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 --2- 172.21.15.50:0/1769360194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b300b3090 0x7f1b300d30e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 -- 172.21.15.50:0/1769360194 >> 172.21.15.50:0/1769360194 conn(0x7f1b3009b960 msgr2=0x7f1b300d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:09.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 -- 172.21.15.50:0/1769360194 shutdown_connections 2023-10-19T10:38:09.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.227+0000 7f1b37560640 1 -- 172.21.15.50:0/1769360194 wait complete. 2023-10-19T10:38:09.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 Processor -- start 2023-10-19T10:38:09.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 -- start start 2023-10-19T10:38:09.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b300b3090 0x7f1b301f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b300b38f0 0x7f1b3015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 0x7f1b3015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300ca850 con 0x7f1b30161210 2023-10-19T10:38:09.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b301f2d60 con 0x7f1b300b38f0 2023-10-19T10:38:09.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b37560640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015d350 con 0x7f1b300b3090 2023-10-19T10:38:09.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.229+0000 7f1b35afd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 0x7f1b3015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 0x7f1b3015eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49866/0 (socket says 172.21.15.50:49866) 2023-10-19T10:38:09.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 -- 172.21.15.50:0/2894456570 learned_addr learned my addr 172.21.15.50:0/2894456570 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:09.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b34afb640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b300b38f0 0x7f1b3015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b352fc640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b300b3090 0x7f1b301f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 -- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b300b3090 msgr2=0x7f1b301f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b300b3090 0x7f1b301f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 -- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b300b38f0 msgr2=0x7f1b3015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b300b38f0 0x7f1b3015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 -- 172.21.15.50:0/2894456570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24009400 con 0x7f1b30161210 2023-10-19T10:38:09.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b35afd640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 0x7f1b3015eee0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000c700 tx=0x7f1b2000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000ce10 con 0x7f1b30161210 2023-10-19T10:38:09.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2000a640 con 0x7f1b30161210 2023-10-19T10:38:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b300caaf0 con 0x7f1b30161210 2023-10-19T10:38:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000a9f0 con 0x7f1b30161210 2023-10-19T10:38:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.230+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b300cb0b0 con 0x7f1b30161210 2023-10-19T10:38:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.231+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3004e420 con 0x7f1b30161210 2023-10-19T10:38:09.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.232+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1b2000ab50 con 0x7f1b30161210 2023-10-19T10:38:09.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.232+0000 7f1b2dffb640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b1c06a050 0x7f1b1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.232+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1b20087f50 con 0x7f1b30161210 2023-10-19T10:38:09.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.232+0000 7f1b352fc640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b1c06a050 0x7f1b1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.233+0000 7f1b352fc640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b1c06a050 0x7f1b1c06c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b300c6080 tx=0x7f1b28007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.236+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1b200533c0 con 0x7f1b30161210 2023-10-19T10:38:09.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.410+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b300b86b0 con 0x7f1b30161210 2023-10-19T10:38:09.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.410+0000 7f1b2dffb640 1 -- 172.21.15.50:0/2894456570 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b20018020 con 0x7f1b30161210 2023-10-19T10:38:09.413 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:38:09.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b1c06a050 msgr2=0x7f1b1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b1c06a050 0x7f1b1c06c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b300c6080 tx=0x7f1b28007e80 comp rx=0 tx=0).stop 2023-10-19T10:38:09.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 msgr2=0x7f1b3015eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 0x7f1b3015eee0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000c700 tx=0x7f1b2000cbc0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 shutdown_connections 2023-10-19T10:38:09.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b1c06a050 0x7f1b1c06c510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b30161210 0x7f1b3015eee0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b300b38f0 0x7f1b3015ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 --2- 172.21.15.50:0/2894456570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b300b3090 0x7f1b301f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 >> 172.21.15.50:0/2894456570 conn(0x7f1b3009b960 msgr2=0x7f1b300b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:09.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 shutdown_connections 2023-10-19T10:38:09.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.412+0000 7f1b37560640 1 -- 172.21.15.50:0/2894456570 wait complete. 2023-10-19T10:38:09.425 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:09.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.533+0000 7f4bb6fa0640 1 Processor -- start 2023-10-19T10:38:09.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.533+0000 7f4bb6fa0640 1 -- start start 2023-10-19T10:38:09.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb6fa0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4bb015c430 0x7f4bb00b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb6fa0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4bb00cf340 0x7f4bb00d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.539 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb6fa0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb00d0c90 con 0x7f4bb00d1d30 2023-10-19T10:38:09.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb6fa0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb00d1910 con 0x7f4bb015c430 2023-10-19T10:38:09.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb6fa0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb00b8280 con 0x7f4bb00cf340 2023-10-19T10:38:09.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb5f9e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4bb015c430 0x7f4bb00b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb579d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f4bb00d1d30 0x7f4bb00cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39298/0 (socket says 172.21.15.50:39298) 2023-10-19T10:38:09.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb579d640 1 -- 172.21.15.50:0/3166325652 learned_addr learned my addr 172.21.15.50:0/3166325652 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:09.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb679f640 1 --2- 172.21.15.50:0/3166325652 >> v2:172.21.15.183:3300/0 conn(0x7f4bb00cf340 0x7f4bb00d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/3166325652 >> v2:172.21.15.183:3300/0 conn(0x7f4bb00cf340 msgr2=0x7f4bb00d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:09.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb5f9e640 1 --2- 172.21.15.50:0/3166325652 >> v2:172.21.15.183:3300/0 conn(0x7f4bb00cf340 0x7f4bb00d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/3166325652 >> v1:172.21.15.50:6789/0 conn(0x7f4bb00d1d30 legacy=0x7f4bb00cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/3166325652 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00d0f70 con 0x7f4bb015c430 2023-10-19T10:38:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.534+0000 7f4bb4f9c640 1 -- 172.21.15.50:0/3166325652 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802772348 0 0) 0x7f4bb00d0c90 con 0x7f4bb00d1d30 2023-10-19T10:38:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.535+0000 7f4bb5f9e640 1 --2- 172.21.15.50:0/3166325652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb015c430 0x7f4bb00b7d40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00d04b0 tx=0x7f4bac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425db642235bbfc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.535+0000 7f4bb4f9c640 1 -- 172.21.15.50:0/3166325652 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac02e070 con 0x7f4bb015c430 2023-10-19T10:38:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.535+0000 7f4bb4f9c640 1 -- 172.21.15.50:0/3166325652 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac0315a0 con 0x7f4bb015c430 2023-10-19T10:38:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.535+0000 7f4bb4f9c640 1 -- 172.21.15.50:0/3166325652 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac031930 con 0x7f4bb015c430 2023-10-19T10:38:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.535+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/3166325652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb015c430 msgr2=0x7f4bb00b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.535+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/3166325652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb015c430 0x7f4bb00b7d40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00d04b0 tx=0x7f4bac02b840 comp rx=0 tx=0).stop 2023-10-19T10:38:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.536+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/3166325652 shutdown_connections 2023-10-19T10:38:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.536+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/3166325652 >> v2:172.21.15.183:3300/0 conn(0x7f4bb00cf340 0x7f4bb00d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.536+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/3166325652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb015c430 0x7f4bb00b7d40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.536+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/3166325652 >> 172.21.15.50:0/3166325652 conn(0x7f4bb006a2b0 msgr2=0x7f4bb006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.536+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/3166325652 shutdown_connections 2023-10-19T10:38:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.536+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/3166325652 wait complete. 2023-10-19T10:38:09.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 Processor -- start 2023-10-19T10:38:09.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 -- start start 2023-10-19T10:38:09.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 0x7f4bb0151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00d1d30 0x7f4bb0154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bb015c430 0x7f4bb01547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0155390 con 0x7f4bb00cf340 2023-10-19T10:38:09.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0154dd0 con 0x7f4bb00d1d30 2023-10-19T10:38:09.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.537+0000 7f4bb6fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb01550b0 con 0x7f4bb015c430 2023-10-19T10:38:09.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 0x7f4bb0151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 0x7f4bb0151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49886/0 (socket says 172.21.15.50:49886) 2023-10-19T10:38:09.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/2364782380 learned_addr learned my addr 172.21.15.50:0/2364782380 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:09.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb579d640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00d1d30 0x7f4bb0154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb679f640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bb015c430 0x7f4bb01547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bb015c430 msgr2=0x7f4bb01547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bb015c430 0x7f4bb01547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00d1d30 msgr2=0x7f4bb0154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00d1d30 0x7f4bb0154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 -- 172.21.15.50:0/2364782380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac009400 con 0x7f4bb00cf340 2023-10-19T10:38:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb5f9e640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 0x7f4bb0151720 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009890 tx=0x7f4bac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac032840 con 0x7f4bb00cf340 2023-10-19T10:38:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac0329a0 con 0x7f4bb00cf340 2023-10-19T10:38:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb015dba0 con 0x7f4bb00cf340 2023-10-19T10:38:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac032d50 con 0x7f4bb00cf340 2023-10-19T10:38:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.538+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb0151c60 con 0x7f4bb00cf340 2023-10-19T10:38:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.539+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb004e420 con 0x7f4bb00cf340 2023-10-19T10:38:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.540+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4bac03b3f0 con 0x7f4bb00cf340 2023-10-19T10:38:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.540+0000 7f4ba67fc640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4b9406a050 0x7f4b9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.540+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4bac0aa890 con 0x7f4bb00cf340 2023-10-19T10:38:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.540+0000 7f4bb579d640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4b9406a050 0x7f4b9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.541+0000 7f4bb579d640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4b9406a050 0x7f4b9406c510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00b0e80 tx=0x7f4ba0007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.544+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4bac075c40 con 0x7f4bb00cf340 2023-10-19T10:38:09.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.717+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4bb00b8040 con 0x7f4bb00cf340 2023-10-19T10:38:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.717+0000 7f4ba67fc640 1 -- 172.21.15.50:0/2364782380 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4bac079260 con 0x7f4bb00cf340 2023-10-19T10:38:09.720 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4b9406a050 msgr2=0x7f4b9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4b9406a050 0x7f4b9406c510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00b0e80 tx=0x7f4ba0007b50 comp rx=0 tx=0).stop 2023-10-19T10:38:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 msgr2=0x7f4bb0151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 0x7f4bb0151720 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009890 tx=0x7f4bac0046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 shutdown_connections 2023-10-19T10:38:09.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f4b9406a050 0x7f4b9406c510 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bb015c430 0x7f4bb01547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00d1d30 0x7f4bb0154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 --2- 172.21.15.50:0/2364782380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bb00cf340 0x7f4bb0151720 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 >> 172.21.15.50:0/2364782380 conn(0x7f4bb006a2b0 msgr2=0x7f4bb00ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 shutdown_connections 2023-10-19T10:38:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.719+0000 7f4bb6fa0640 1 -- 172.21.15.50:0/2364782380 wait complete. 2023-10-19T10:38:09.733 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:38:09.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.837+0000 7f486bfa7640 1 Processor -- start 2023-10-19T10:38:09.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f486bfa7640 1 -- start start 2023-10-19T10:38:09.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f486bfa7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f48640d0e30 0x7f486415c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f486bfa7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486406aa30 con 0x7f486415c570 2023-10-19T10:38:09.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f486bfa7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486406b2d0 con 0x7f48640cd920 2023-10-19T10:38:09.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f486bfa7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486415ce90 con 0x7f48640d0e30 2023-10-19T10:38:09.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f4869542640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f48640d0e30 0x7f486415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f4869d43640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f486415c570 0x7f486406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39314/0 (socket says 172.21.15.50:39314) 2023-10-19T10:38:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f4869542640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f48640d0e30 0x7f486415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49892/0 (socket says 172.21.15.50:49892) 2023-10-19T10:38:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.838+0000 7f4869d43640 1 -- 172.21.15.50:0/941637733 learned_addr learned my addr 172.21.15.50:0/941637733 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.839+0000 7f4869542640 1 -- 172.21.15.50:0/941637733 >> v1:172.21.15.183:6789/0 conn(0x7f48640cd920 legacy=0x7f48640d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.839+0000 7f4869542640 1 -- 172.21.15.50:0/941637733 >> v1:172.21.15.50:6789/0 conn(0x7f486415c570 legacy=0x7f486406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.839+0000 7f4868d41640 1 -- 172.21.15.50:0/941637733 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912576887 0 0) 0x7f486406aa30 con 0x7f486415c570 2023-10-19T10:38:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.839+0000 7f4869542640 1 -- 172.21.15.50:0/941637733 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48640d1270 con 0x7f48640d0e30 2023-10-19T10:38:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f4869542640 1 --2- 172.21.15.50:0/941637733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f486415c950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f48540097a0 tx=0x7f485402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b548c7caa82964 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f4868d41640 1 -- 172.21.15.50:0/941637733 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854031830 con 0x7f48640d0e30 2023-10-19T10:38:09.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f4868d41640 1 -- 172.21.15.50:0/941637733 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854031990 con 0x7f48640d0e30 2023-10-19T10:38:09.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f4868d41640 1 -- 172.21.15.50:0/941637733 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854031d20 con 0x7f48640d0e30 2023-10-19T10:38:09.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 -- 172.21.15.50:0/941637733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 msgr2=0x7f486415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 --2- 172.21.15.50:0/941637733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f486415c950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f48540097a0 tx=0x7f485402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 -- 172.21.15.50:0/941637733 shutdown_connections 2023-10-19T10:38:09.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 --2- 172.21.15.50:0/941637733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f486415c950 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 -- 172.21.15.50:0/941637733 >> 172.21.15.50:0/941637733 conn(0x7f4864089ff0 msgr2=0x7f486409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:09.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 -- 172.21.15.50:0/941637733 shutdown_connections 2023-10-19T10:38:09.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.840+0000 7f486bfa7640 1 -- 172.21.15.50:0/941637733 wait complete. 2023-10-19T10:38:09.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.841+0000 7f486bfa7640 1 Processor -- start 2023-10-19T10:38:09.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.841+0000 7f486bfa7640 1 -- start start 2023-10-19T10:38:09.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f486bfa7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48640cd920 0x7f48641f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f486bfa7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f48640c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f486bfa7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f486415c570 0x7f48640d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f486bfa7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48640ca3f0 con 0x7f48640d0e30 2023-10-19T10:38:09.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f486bfa7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48640c97e0 con 0x7f486415c570 2023-10-19T10:38:09.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f486bfa7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48640c9ac0 con 0x7f48640cd920 2023-10-19T10:38:09.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f4869542640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f48640c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f4869542640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f48640c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49894/0 (socket says 172.21.15.50:49894) 2023-10-19T10:38:09.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.842+0000 7f4869542640 1 -- 172.21.15.50:0/693223035 learned_addr learned my addr 172.21.15.50:0/693223035 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:09.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869d43640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48640cd920 0x7f48641f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f486a544640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f486415c570 0x7f48640d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869542640 1 -- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48640cd920 msgr2=0x7f48641f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869542640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48640cd920 0x7f48641f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869542640 1 -- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f486415c570 msgr2=0x7f48640d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:09.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869542640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f486415c570 0x7f48640d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:09.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869542640 1 -- 172.21.15.50:0/693223035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009400 con 0x7f48640d0e30 2023-10-19T10:38:09.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f4869542640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f48640c8510 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f48540098c0 tx=0x7f48540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485403ba70 con 0x7f48640d0e30 2023-10-19T10:38:09.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485403bbf0 con 0x7f48640d0e30 2023-10-19T10:38:09.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48640c9da0 con 0x7f48640d0e30 2023-10-19T10:38:09.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540453f0 con 0x7f48640d0e30 2023-10-19T10:38:09.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.843+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48641f2060 con 0x7f48640d0e30 2023-10-19T10:38:09.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.844+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4864073990 con 0x7f48640d0e30 2023-10-19T10:38:09.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.844+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f485404e450 con 0x7f48640d0e30 2023-10-19T10:38:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.845+0000 7f485a7fc640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f484806a050 0x7f484806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.845+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f48540aabb0 con 0x7f48640d0e30 2023-10-19T10:38:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.845+0000 7f4869d43640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f484806a050 0x7f484806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.846+0000 7f4869d43640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f484806a050 0x7f484806c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4860007fd0 tx=0x7f4860009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:09.849+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4854075f70 con 0x7f48640d0e30 2023-10-19T10:38:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.037+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f48640d3480 con 0x7f48640d0e30 2023-10-19T10:38:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.037+0000 7f485a7fc640 1 -- 172.21.15.50:0/693223035 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4854035080 con 0x7f48640d0e30 2023-10-19T10:38:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:38:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f484806a050 msgr2=0x7f484806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f484806a050 0x7f484806c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4860007fd0 tx=0x7f4860009040 comp rx=0 tx=0).stop 2023-10-19T10:38:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 msgr2=0x7f48640c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f48640c8510 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f48540098c0 tx=0x7f48540046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 shutdown_connections 2023-10-19T10:38:10.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f484806a050 0x7f484806c510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f486415c570 0x7f48640d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48640d0e30 0x7f48640c8510 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 --2- 172.21.15.50:0/693223035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48640cd920 0x7f48641f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 >> 172.21.15.50:0/693223035 conn(0x7f4864089ff0 msgr2=0x7f48640b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 shutdown_connections 2023-10-19T10:38:10.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.039+0000 7f486bfa7640 1 -- 172.21.15.50:0/693223035 wait complete. 2023-10-19T10:38:10.052 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:38:10.053 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:38:10.072 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:10.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.176+0000 7fed0c66a640 1 Processor -- start 2023-10-19T10:38:10.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.176+0000 7fed0c66a640 1 -- start start 2023-10-19T10:38:10.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0c66a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed041609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0c66a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fed04160ee0 0x7fed040ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0c66a640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0415d450 con 0x7fed0415a290 2023-10-19T10:38:10.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0c66a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0406a530 con 0x7fed04160ee0 2023-10-19T10:38:10.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0c66a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0415a670 con 0x7fed040cbb20 2023-10-19T10:38:10.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fed04160ee0 0x7fed040ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fed04160ee0 0x7fed040ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49902/0 (socket says 172.21.15.50:49902) 2023-10-19T10:38:10.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 -- 172.21.15.50:0/2744155738 learned_addr learned my addr 172.21.15.50:0/2744155738 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:10.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0a406640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fed0415a290 0x7fed0415cbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39326/0 (socket says 172.21.15.50:39326) 2023-10-19T10:38:10.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed09c05640 1 --2- 172.21.15.50:0/2744155738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed041609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 -- 172.21.15.50:0/2744155738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 msgr2=0x7fed041609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:10.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 --2- 172.21.15.50:0/2744155738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed041609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 -- 172.21.15.50:0/2744155738 >> v1:172.21.15.50:6789/0 conn(0x7fed0415a290 legacy=0x7fed0415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed09404640 1 -- 172.21.15.50:0/2744155738 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083158862 0 0) 0x7fed0415d450 con 0x7fed0415a290 2023-10-19T10:38:10.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.177+0000 7fed0ac07640 1 -- 172.21.15.50:0/2744155738 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0415d730 con 0x7fed04160ee0 2023-10-19T10:38:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.178+0000 7fed0ac07640 1 --2- 172.21.15.50:0/2744155738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040ccd60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009e50 tx=0x7fecf402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5452d5c44356b565 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.178+0000 7fed09404640 1 -- 172.21.15.50:0/2744155738 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf402cb00 con 0x7fed04160ee0 2023-10-19T10:38:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.178+0000 7fed09404640 1 -- 172.21.15.50:0/2744155738 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf402cc60 con 0x7fed04160ee0 2023-10-19T10:38:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.178+0000 7fed09404640 1 -- 172.21.15.50:0/2744155738 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40343f0 con 0x7fed04160ee0 2023-10-19T10:38:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 -- 172.21.15.50:0/2744155738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 msgr2=0x7fed040ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 --2- 172.21.15.50:0/2744155738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040ccd60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009e50 tx=0x7fecf402bf60 comp rx=0 tx=0).stop 2023-10-19T10:38:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 -- 172.21.15.50:0/2744155738 shutdown_connections 2023-10-19T10:38:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 --2- 172.21.15.50:0/2744155738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040ccd60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 --2- 172.21.15.50:0/2744155738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed041609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 -- 172.21.15.50:0/2744155738 >> 172.21.15.50:0/2744155738 conn(0x7fed0406c9a0 msgr2=0x7fed0406cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 -- 172.21.15.50:0/2744155738 shutdown_connections 2023-10-19T10:38:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.179+0000 7fed0c66a640 1 -- 172.21.15.50:0/2744155738 wait complete. 2023-10-19T10:38:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.180+0000 7fed0c66a640 1 Processor -- start 2023-10-19T10:38:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.180+0000 7fed0c66a640 1 -- start start 2023-10-19T10:38:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0c66a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed0415b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0c66a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0415a290 0x7fed040c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0c66a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0c66a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0415bd40 con 0x7fed04160ee0 2023-10-19T10:38:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0c66a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04149da0 con 0x7fed0415a290 2023-10-19T10:38:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0c66a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed04149190 con 0x7fed040cbb20 2023-10-19T10:38:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49916/0 (socket says 172.21.15.50:49916) 2023-10-19T10:38:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 -- 172.21.15.50:0/4265685667 learned_addr learned my addr 172.21.15.50:0/4265685667 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed09c05640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0415a290 0x7fed040c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0a406640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed0415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 -- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 msgr2=0x7fed0415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed0415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 -- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0415a290 msgr2=0x7fed040c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0415a290 0x7fed040c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 -- 172.21.15.50:0/4265685667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4009b30 con 0x7fed04160ee0 2023-10-19T10:38:10.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed0ac07640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040adcc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009e50 tx=0x7fecf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.181+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf403e450 con 0x7fed04160ee0 2023-10-19T10:38:10.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.182+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf403e5d0 con 0x7fed04160ee0 2023-10-19T10:38:10.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.182+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed04149470 con 0x7fed04160ee0 2023-10-19T10:38:10.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.182+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf403e780 con 0x7fed04160ee0 2023-10-19T10:38:10.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.182+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04149a30 con 0x7fed04160ee0 2023-10-19T10:38:10.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.183+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed04149f30 con 0x7fed04160ee0 2023-10-19T10:38:10.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.183+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fecf403e8e0 con 0x7fed04160ee0 2023-10-19T10:38:10.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.183+0000 7fed02ffd640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fecf006a050 0x7fecf006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.183+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fecf40ab410 con 0x7fed04160ee0 2023-10-19T10:38:10.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.184+0000 7fed0a406640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fecf006a050 0x7fecf006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.184+0000 7fed0a406640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fecf006a050 0x7fecf006c510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000f60 tx=0x7fecfc007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.188+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fecf40767d0 con 0x7fed04160ee0 2023-10-19T10:38:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.364+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed04073990 con 0x7fed04160ee0 2023-10-19T10:38:10.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.365+0000 7fed02ffd640 1 -- 172.21.15.50:0/4265685667 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7fecf4079df0 con 0x7fed04160ee0 2023-10-19T10:38:10.370 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:10.370 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:10.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fecf006a050 msgr2=0x7fecf006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fecf006a050 0x7fecf006c510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000f60 tx=0x7fecfc007c00 comp rx=0 tx=0).stop 2023-10-19T10:38:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 msgr2=0x7fed040adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040adcc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009e50 tx=0x7fecf4004a40 comp rx=0 tx=0).stop 2023-10-19T10:38:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 shutdown_connections 2023-10-19T10:38:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fecf006a050 0x7fecf006c510 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed04160ee0 0x7fed040adcc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0415a290 0x7fed040c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 --2- 172.21.15.50:0/4265685667 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed040cbb20 0x7fed0415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 >> 172.21.15.50:0/4265685667 conn(0x7fed0406c9a0 msgr2=0x7fed041576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 shutdown_connections 2023-10-19T10:38:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.367+0000 7fed0c66a640 1 -- 172.21.15.50:0/4265685667 wait complete. 2023-10-19T10:38:10.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:38:10.382 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:10.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.489+0000 7faa0391f640 1 Processor -- start 2023-10-19T10:38:10.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.489+0000 7faa0391f640 1 -- start start 2023-10-19T10:38:10.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa0391f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9fc15c5c0 0x7fa9fc06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa0391f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc15d8f0 con 0x7fa9fc0b3f50 2023-10-19T10:38:10.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa0391f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc0d0a50 con 0x7fa9fc15d460 2023-10-19T10:38:10.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa0391f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d1010 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa016bb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9fc15c5c0 0x7fa9fc06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa016bb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa9fc15c5c0 0x7fa9fc06b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49930/0 (socket says 172.21.15.50:49930) 2023-10-19T10:38:10.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa016bb640 1 -- 172.21.15.50:0/1194885938 learned_addr learned my addr 172.21.15.50:0/1194885938 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:10.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa01ebc640 1 --1- 172.21.15.50:0/1194885938 >> v1:172.21.15.50:6789/0 conn(0x7fa9fc0b3f50 0x7fa9fc0ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39340/0 (socket says 172.21.15.50:39340) 2023-10-19T10:38:10.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa016bb640 1 -- 172.21.15.50:0/1194885938 >> v1:172.21.15.183:6789/0 conn(0x7fa9fc15d460 legacy=0x7fa9fc0d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa016bb640 1 -- 172.21.15.50:0/1194885938 >> v1:172.21.15.50:6789/0 conn(0x7fa9fc0b3f50 legacy=0x7fa9fc0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7fa9f3fff640 1 -- 172.21.15.50:0/1194885938 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166710459 0 0) 0x7fa9fc15d8f0 con 0x7fa9fc0b3f50 2023-10-19T10:38:10.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.490+0000 7faa016bb640 1 -- 172.21.15.50:0/1194885938 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0d0d30 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.491+0000 7faa016bb640 1 --2- 172.21.15.50:0/1194885938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc06b1f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009870 tx=0x7fa9f802bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5ea86ced8fea11 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.491+0000 7fa9f3fff640 1 -- 172.21.15.50:0/1194885938 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802d870 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.491+0000 7fa9f3fff640 1 -- 172.21.15.50:0/1194885938 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f802d9d0 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.491+0000 7fa9f3fff640 1 -- 172.21.15.50:0/1194885938 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802dd60 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 -- 172.21.15.50:0/1194885938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 msgr2=0x7fa9fc06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 --2- 172.21.15.50:0/1194885938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc06b1f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009870 tx=0x7fa9f802bab0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 -- 172.21.15.50:0/1194885938 shutdown_connections 2023-10-19T10:38:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 --2- 172.21.15.50:0/1194885938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc06b1f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 -- 172.21.15.50:0/1194885938 >> 172.21.15.50:0/1194885938 conn(0x7fa9fc071350 msgr2=0x7fa9fc071760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 -- 172.21.15.50:0/1194885938 shutdown_connections 2023-10-19T10:38:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.492+0000 7faa0391f640 1 -- 172.21.15.50:0/1194885938 wait complete. 2023-10-19T10:38:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.493+0000 7faa0391f640 1 Processor -- start 2023-10-19T10:38:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.493+0000 7faa0391f640 1 -- start start 2023-10-19T10:38:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9fc0b3f50 0x7fa9fc0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9fc15d460 0x7fa9fc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0c9ab0 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc158c20 con 0x7fa9fc0b3f50 2023-10-19T10:38:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc158660 con 0x7fa9fc15d460 2023-10-19T10:38:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49944/0 (socket says 172.21.15.50:49944) 2023-10-19T10:38:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 -- 172.21.15.50:0/4044218816 learned_addr learned my addr 172.21.15.50:0/4044218816 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa01ebc640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9fc15d460 0x7fa9fc155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa016bb640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9fc0b3f50 0x7fa9fc0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 -- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9fc15d460 msgr2=0x7fa9fc155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9fc15d460 0x7fa9fc155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 -- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9fc0b3f50 msgr2=0x7fa9fc0b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9fc0b3f50 0x7fa9fc0b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 -- 172.21.15.50:0/4044218816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80094d0 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa00eba640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc158050 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c400 tx=0x7fa9ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0052d0 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec005430 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc158940 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.494+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec003830 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.495+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc155f50 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.496+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9fc04e420 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.496+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9ec003990 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.496+0000 7fa9f1ffb640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9e006a080 0x7fa9e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.496+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa9ec087dd0 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.497+0000 7faa016bb640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9e006a080 0x7fa9e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.497+0000 7faa016bb640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9e006a080 0x7fa9e006c540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8036000 tx=0x7fa9f8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.507 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.501+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa9ec053220 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.676+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9fc0cdfc0 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.676+0000 7fa9f1ffb640 1 -- 172.21.15.50:0/4044218816 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9ec017020 con 0x7fa9fc15c5c0 2023-10-19T10:38:10.679 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:10.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9e006a080 msgr2=0x7fa9e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9e006a080 0x7fa9e006c540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8036000 tx=0x7fa9f8008030 comp rx=0 tx=0).stop 2023-10-19T10:38:10.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 msgr2=0x7fa9fc158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc158050 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c400 tx=0x7fa9ec00c8c0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 shutdown_connections 2023-10-19T10:38:10.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa9e006a080 0x7fa9e006c540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9fc15d460 0x7fa9fc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9fc15c5c0 0x7fa9fc158050 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 --2- 172.21.15.50:0/4044218816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9fc0b3f50 0x7fa9fc0b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 >> 172.21.15.50:0/4044218816 conn(0x7fa9fc071350 msgr2=0x7fa9fc0ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 shutdown_connections 2023-10-19T10:38:10.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.679+0000 7faa0391f640 1 -- 172.21.15.50:0/4044218816 wait complete. 2023-10-19T10:38:10.692 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:10.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.798+0000 7f2651399640 1 Processor -- start 2023-10-19T10:38:10.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.798+0000 7f2651399640 1 -- start start 2023-10-19T10:38:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f2651399640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f264c161210 0x7f264c0d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f2651399640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c0af250 con 0x7f264c0b3090 2023-10-19T10:38:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f2651399640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c0af510 con 0x7f264c0b38f0 2023-10-19T10:38:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f2651399640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264c0b0190 con 0x7f264c161210 2023-10-19T10:38:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f264b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f264c161210 0x7f264c0d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f264b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f264c161210 0x7f264c0d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49948/0 (socket says 172.21.15.50:49948) 2023-10-19T10:38:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f264b7fe640 1 -- 172.21.15.50:0/2558766759 learned_addr learned my addr 172.21.15.50:0/2558766759 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f264b7fe640 1 -- 172.21.15.50:0/2558766759 >> v1:172.21.15.183:6789/0 conn(0x7f264c0b38f0 legacy=0x7f264c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f264b7fe640 1 -- 172.21.15.50:0/2558766759 >> v1:172.21.15.50:6789/0 conn(0x7f264c0b3090 legacy=0x7f264c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f264b7fe640 1 -- 172.21.15.50:0/2558766759 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c0af7f0 con 0x7f264c161210 2023-10-19T10:38:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.799+0000 7f2649ffb640 1 -- 172.21.15.50:0/2558766759 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412695931 0 0) 0x7f264c0af250 con 0x7f264c0b3090 2023-10-19T10:38:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.800+0000 7f264b7fe640 1 --2- 172.21.15.50:0/2558766759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c0d30e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f263c009e50 tx=0x7f263c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f34c2cce4fc59f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.800+0000 7f2649ffb640 1 -- 172.21.15.50:0/2558766759 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c02cb20 con 0x7f264c161210 2023-10-19T10:38:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.800+0000 7f2649ffb640 1 -- 172.21.15.50:0/2558766759 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c02cc80 con 0x7f264c161210 2023-10-19T10:38:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.800+0000 7f2649ffb640 1 -- 172.21.15.50:0/2558766759 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0343f0 con 0x7f264c161210 2023-10-19T10:38:10.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 -- 172.21.15.50:0/2558766759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 msgr2=0x7f264c0d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 --2- 172.21.15.50:0/2558766759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c0d30e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f263c009e50 tx=0x7f263c02bf80 comp rx=0 tx=0).stop 2023-10-19T10:38:10.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 -- 172.21.15.50:0/2558766759 shutdown_connections 2023-10-19T10:38:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 --2- 172.21.15.50:0/2558766759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c0d30e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 -- 172.21.15.50:0/2558766759 >> 172.21.15.50:0/2558766759 conn(0x7f264c09b960 msgr2=0x7f264c0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 -- 172.21.15.50:0/2558766759 shutdown_connections 2023-10-19T10:38:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.801+0000 7f2651399640 1 -- 172.21.15.50:0/2558766759 wait complete. 2023-10-19T10:38:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.802+0000 7f2651399640 1 Processor -- start 2023-10-19T10:38:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.802+0000 7f2651399640 1 -- start start 2023-10-19T10:38:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f2651399640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f264c0b3090 0x7f264c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f2651399640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264c0b38f0 0x7f264c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f2651399640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f2651399640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0cf3b0 con 0x7f264c161210 2023-10-19T10:38:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f2651399640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0cf650 con 0x7f264c0b38f0 2023-10-19T10:38:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f2651399640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0cf930 con 0x7f264c0b3090 2023-10-19T10:38:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49958/0 (socket says 172.21.15.50:49958) 2023-10-19T10:38:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 -- 172.21.15.50:0/522806695 learned_addr learned my addr 172.21.15.50:0/522806695 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 -- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f264c0b3090 msgr2=0x7f264c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f264c0b3090 0x7f264c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 -- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264c0b38f0 msgr2=0x7f264c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264a7fc640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264c0b38f0 0x7f264c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264c0b38f0 0x7f264c0cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264b7fe640 1 -- 172.21.15.50:0/522806695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c009b30 con 0x7f264c161210 2023-10-19T10:38:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.803+0000 7f264a7fc640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264c0b38f0 0x7f264c0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.804+0000 7f264b7fe640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c159490 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f263c009e50 tx=0x7f263c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.804+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c009510 con 0x7f264c161210 2023-10-19T10:38:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.804+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c03e450 con 0x7f264c161210 2023-10-19T10:38:10.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.804+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c0b61c0 con 0x7f264c161210 2023-10-19T10:38:10.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.804+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c03e600 con 0x7f264c161210 2023-10-19T10:38:10.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.804+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c0b6780 con 0x7f264c161210 2023-10-19T10:38:10.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.805+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c0b6020 con 0x7f264c161210 2023-10-19T10:38:10.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.805+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f263c03e760 con 0x7f264c161210 2023-10-19T10:38:10.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.806+0000 7f26437fe640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f262c06a050 0x7f262c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:10.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.806+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f263c0ab1b0 con 0x7f264c161210 2023-10-19T10:38:10.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.806+0000 7f264affd640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f262c06a050 0x7f262c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:10.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.807+0000 7f264affd640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f262c06a050 0x7f262c06c510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2644003d10 tx=0x7f2644007a00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:10.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.810+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f264c0b6020 con 0x7f264c161210 2023-10-19T10:38:10.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.983+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f264c0b6020 con 0x7f264c161210 2023-10-19T10:38:10.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.984+0000 7f26437fe640 1 -- 172.21.15.50:0/522806695 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f264c0b6020 con 0x7f264c161210 2023-10-19T10:38:10.987 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:10.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f262c06a050 msgr2=0x7f262c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f262c06a050 0x7f262c06c510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2644003d10 tx=0x7f2644007a00 comp rx=0 tx=0).stop 2023-10-19T10:38:10.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 msgr2=0x7f264c159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:10.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c159490 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f263c009e50 tx=0x7f263c004a40 comp rx=0 tx=0).stop 2023-10-19T10:38:10.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 shutdown_connections 2023-10-19T10:38:10.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f262c06a050 0x7f262c06c510 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f264c161210 0x7f264c159490 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264c0b38f0 0x7f264c0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 --2- 172.21.15.50:0/522806695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f264c0b3090 0x7f264c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:10.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 >> 172.21.15.50:0/522806695 conn(0x7f264c09b960 msgr2=0x7f264c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:10.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 shutdown_connections 2023-10-19T10:38:10.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:10.986+0000 7f2651399640 1 -- 172.21.15.50:0/522806695 wait complete. 2023-10-19T10:38:10.999 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:38:11.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.106+0000 7fa63a09d640 1 Processor -- start 2023-10-19T10:38:11.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.106+0000 7fa63a09d640 1 -- start start 2023-10-19T10:38:11.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.106+0000 7fa63a09d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa6340ce730 0x7fa63406f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.106+0000 7fa63a09d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 0x7fa63406aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.106+0000 7fa63a09d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6340701b0 con 0x7fa6340b26e0 2023-10-19T10:38:11.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa63a09d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63406f910 con 0x7fa6340ce730 2023-10-19T10:38:11.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa63a09d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63406fbf0 con 0x7fa634145880 2023-10-19T10:38:11.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6337fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa6340b26e0 0x7fa6340ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39364/0 (socket says 172.21.15.50:39364) 2023-10-19T10:38:11.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6337fe640 1 -- 172.21.15.50:0/100201968 learned_addr learned my addr 172.21.15.50:0/100201968 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:11.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa633fff640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 0x7fa63406aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa632ffd640 1 --2- 172.21.15.50:0/100201968 >> v2:172.21.15.183:3300/0 conn(0x7fa6340ce730 0x7fa63406f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557867192 0 0) 0x7fa6340701b0 con 0x7fa6340b26e0 2023-10-19T10:38:11.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa61c003520 con 0x7fa6340b26e0 2023-10-19T10:38:11.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3404998110 0 0) 0x7fa61c003520 con 0x7fa6340b26e0 2023-10-19T10:38:11.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 msgr2=0x7fa63406aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 0x7fa63406aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 >> v2:172.21.15.183:3300/0 conn(0x7fa6340ce730 msgr2=0x7fa63406f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 --2- 172.21.15.50:0/100201968 >> v2:172.21.15.183:3300/0 conn(0x7fa6340ce730 0x7fa63406f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63406fed0 con 0x7fa6340b26e0 2023-10-19T10:38:11.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fa624001670 con 0x7fa6340b26e0 2023-10-19T10:38:11.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 >> v1:172.21.15.50:6789/0 conn(0x7fa6340b26e0 legacy=0x7fa6340ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c003970 0x7fa61c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c0047c0 0x7fa61c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 0x7fa61c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63406f910 con 0x7fa61c003970 2023-10-19T10:38:11.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63406fbf0 con 0x7fa61c0047c0 2023-10-19T10:38:11.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c00b980 con 0x7fa634145880 2023-10-19T10:38:11.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.107+0000 7fa6337fe640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c003970 0x7fa61c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa632ffd640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c0047c0 0x7fa61c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa633fff640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 0x7fa61c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa6337fe640 1 -- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 msgr2=0x7fa61c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa6337fe640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa634145880 0x7fa61c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa6337fe640 1 -- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c0047c0 msgr2=0x7fa61c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa6337fe640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c0047c0 0x7fa61c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.108+0000 7fa6337fe640 1 -- 172.21.15.50:0/100201968 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c003520 con 0x7fa61c003970 2023-10-19T10:38:11.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.109+0000 7fa6337fe640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c003970 0x7fa61c003d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c2c0 tx=0x7fa61c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92cc412ef4931a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.109+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624005ad0 con 0x7fa61c003970 2023-10-19T10:38:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.109+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624005c30 con 0x7fa61c003970 2023-10-19T10:38:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.109+0000 7fa6327fc640 1 -- 172.21.15.50:0/100201968 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62402c8b0 con 0x7fa61c003970 2023-10-19T10:38:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa63a09d640 1 -- 172.21.15.50:0/100201968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c003970 msgr2=0x7fa61c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa63a09d640 1 --2- 172.21.15.50:0/100201968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c003970 0x7fa61c003d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c2c0 tx=0x7fa61c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa633fff640 1 -- 172.21.15.50:0/100201968 reap_dead start 2023-10-19T10:38:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa63a09d640 1 -- 172.21.15.50:0/100201968 shutdown_connections 2023-10-19T10:38:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa63a09d640 1 -- 172.21.15.50:0/100201968 >> 172.21.15.50:0/100201968 conn(0x7fa6340ccf60 msgr2=0x7fa6340cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa63a09d640 1 -- 172.21.15.50:0/100201968 shutdown_connections 2023-10-19T10:38:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.110+0000 7fa63a09d640 1 -- 172.21.15.50:0/100201968 wait complete. 2023-10-19T10:38:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.111+0000 7fa63a09d640 1 Processor -- start 2023-10-19T10:38:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.111+0000 7fa63a09d640 1 -- start start 2023-10-19T10:38:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa63a09d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c003970 0x7fa6340c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa63a09d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 0x7fa6341f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa63a09d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6340b46f0 0x7fa6340b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa63a09d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340b50e0 con 0x7fa6340b5d70 2023-10-19T10:38:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa63a09d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340c4700 con 0x7fa61c003970 2023-10-19T10:38:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa63a09d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340c7110 con 0x7fa6340b46f0 2023-10-19T10:38:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 0x7fa6341f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 0x7fa6341f1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49986/0 (socket says 172.21.15.50:49986) 2023-10-19T10:38:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 -- 172.21.15.50:0/1582141189 learned_addr learned my addr 172.21.15.50:0/1582141189 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa6337fe640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c003970 0x7fa6340c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa633fff640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6340b46f0 0x7fa6340b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 -- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6340b46f0 msgr2=0x7fa6340b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6340b46f0 0x7fa6340b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 -- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c003970 msgr2=0x7fa6340c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c003970 0x7fa6340c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa632ffd640 1 -- 172.21.15.50:0/1582141189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340ad0f0 con 0x7fa6340b5d70 2023-10-19T10:38:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.112+0000 7fa633fff640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6340b46f0 0x7fa6340b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.113+0000 7fa632ffd640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 0x7fa6341f1680 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa628011910 tx=0x7fa628011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.113+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800faf0 con 0x7fa6340b5d70 2023-10-19T10:38:11.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.113+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800fc50 con 0x7fa6340b5d70 2023-10-19T10:38:11.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.113+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6340c6e30 con 0x7fa6340b5d70 2023-10-19T10:38:11.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.113+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628014440 con 0x7fa6340b5d70 2023-10-19T10:38:11.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.113+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6340c4390 con 0x7fa6340b5d70 2023-10-19T10:38:11.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.114+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63404e420 con 0x7fa6340b5d70 2023-10-19T10:38:11.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.114+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa6280145a0 con 0x7fa6340b5d70 2023-10-19T10:38:11.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.115+0000 7fa61bfff640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa62006a050 0x7fa62006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.115+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa628088ea0 con 0x7fa6340b5d70 2023-10-19T10:38:11.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.115+0000 7fa6337fe640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa62006a050 0x7fa62006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.115+0000 7fa6337fe640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa62006a050 0x7fa62006c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa624034000 tx=0x7fa624007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.119+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa6280542f0 con 0x7fa6340b5d70 2023-10-19T10:38:11.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.293+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6340b6460 con 0x7fa6340b5d70 2023-10-19T10:38:11.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.293+0000 7fa61bfff640 1 -- 172.21.15.50:0/1582141189 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa628057910 con 0x7fa6340b5d70 2023-10-19T10:38:11.296 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:11.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa62006a050 msgr2=0x7fa62006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa62006a050 0x7fa62006c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa624034000 tx=0x7fa624007000 comp rx=0 tx=0).stop 2023-10-19T10:38:11.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 msgr2=0x7fa6341f1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 0x7fa6341f1680 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa628011910 tx=0x7fa628011dd0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 shutdown_connections 2023-10-19T10:38:11.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa62006a050 0x7fa62006c510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6340b46f0 0x7fa6340b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.299 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6340b5d70 0x7fa6341f1680 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 --2- 172.21.15.50:0/1582141189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa61c003970 0x7fa6340c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 >> 172.21.15.50:0/1582141189 conn(0x7fa6340ccf60 msgr2=0x7fa6340ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:11.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 shutdown_connections 2023-10-19T10:38:11.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.295+0000 7fa63a09d640 1 -- 172.21.15.50:0/1582141189 wait complete. 2023-10-19T10:38:11.308 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.418+0000 7f3f4deee640 1 Processor -- start 2023-10-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.418+0000 7f3f4deee640 1 -- start start 2023-10-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f4deee640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3f480b4070 0x7f3f480b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f4deee640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3f4815c960 0x7f3f480d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f4deee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f4815cdf0 0x7f3f48160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f4deee640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480b4990 con 0x7f3f4815c960 2023-10-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f4deee640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4815d280 con 0x7f3f480b4070 2023-10-19T10:38:11.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f4deee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4815d560 con 0x7f3f4815cdf0 2023-10-19T10:38:11.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3f4815c960 0x7f3f480d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3f4815c960 0x7f3f480d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49988/0 (socket says 172.21.15.50:49988) 2023-10-19T10:38:11.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 -- 172.21.15.50:0/2901087961 learned_addr learned my addr 172.21.15.50:0/2901087961 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:11.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f47fff640 1 --2- 172.21.15.50:0/2901087961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f4815cdf0 0x7f3f48160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2901087961 >> v2:172.21.15.183:3300/0 conn(0x7f3f480b4070 0x7f3f480b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 -- 172.21.15.50:0/2901087961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f4815cdf0 msgr2=0x7f3f48160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2901087961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f4815cdf0 0x7f3f48160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 -- 172.21.15.50:0/2901087961 >> v2:172.21.15.183:3300/0 conn(0x7f3f480b4070 msgr2=0x7f3f480b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2901087961 >> v2:172.21.15.183:3300/0 conn(0x7f3f480b4070 0x7f3f480b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f46ffd640 1 -- 172.21.15.50:0/2901087961 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4815d840 con 0x7f3f4815c960 2023-10-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.419+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2901087961 >> v2:172.21.15.183:3300/0 conn(0x7f3f480b4070 0x7f3f480b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.420+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2901087961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f4815c960 0x7f3f480d2d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3f4808a3b0 tx=0x7f3f3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a57a8aa78c9c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.420+0000 7f3f467fc640 1 -- 172.21.15.50:0/2901087961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c031870 con 0x7f3f4815c960 2023-10-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.420+0000 7f3f467fc640 1 -- 172.21.15.50:0/2901087961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0319d0 con 0x7f3f4815c960 2023-10-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.420+0000 7f3f467fc640 1 -- 172.21.15.50:0/2901087961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c031d60 con 0x7f3f4815c960 2023-10-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 -- 172.21.15.50:0/2901087961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f4815c960 msgr2=0x7f3f480d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2901087961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f4815c960 0x7f3f480d2d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3f4808a3b0 tx=0x7f3f3c02b910 comp rx=0 tx=0).stop 2023-10-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 -- 172.21.15.50:0/2901087961 shutdown_connections 2023-10-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2901087961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f4815cdf0 0x7f3f48160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2901087961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f4815c960 0x7f3f480d2d70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2901087961 >> v2:172.21.15.183:3300/0 conn(0x7f3f480b4070 0x7f3f480b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 -- 172.21.15.50:0/2901087961 >> 172.21.15.50:0/2901087961 conn(0x7f3f4809b970 msgr2=0x7f3f480d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 -- 172.21.15.50:0/2901087961 shutdown_connections 2023-10-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.421+0000 7f3f4deee640 1 -- 172.21.15.50:0/2901087961 wait complete. 2023-10-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.422+0000 7f3f4deee640 1 Processor -- start 2023-10-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.422+0000 7f3f4deee640 1 -- start start 2023-10-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f4deee640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 0x7f3f48155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f4deee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4815c960 0x7f3f480c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f4deee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f480c7920 0x7f3f48145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f4deee640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480c80c0 con 0x7f3f480b4070 2023-10-19T10:38:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f4deee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480c83a0 con 0x7f3f4815c960 2023-10-19T10:38:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f4deee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480c8680 con 0x7f3f480c7920 2023-10-19T10:38:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 0x7f3f48155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 0x7f3f48155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49994/0 (socket says 172.21.15.50:49994) 2023-10-19T10:38:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 -- 172.21.15.50:0/2525179734 learned_addr learned my addr 172.21.15.50:0/2525179734 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4815c960 0x7f3f480c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f47fff640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f480c7920 0x7f3f48145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 -- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f480c7920 msgr2=0x7f3f48145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f480c7920 0x7f3f48145830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 -- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4815c960 msgr2=0x7f3f480c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4815c960 0x7f3f480c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f477fe640 1 -- 172.21.15.50:0/2525179734 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0094d0 con 0x7f3f480b4070 2023-10-19T10:38:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.423+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4815c960 0x7f3f480c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:11.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.424+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 0x7f3f48155c20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c7c0 tx=0x7f3f3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.424+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38005b90 con 0x7f3f480b4070 2023-10-19T10:38:11.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.424+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38005cf0 con 0x7f3f480b4070 2023-10-19T10:38:11.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.424+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f480c8960 con 0x7f3f480b4070 2023-10-19T10:38:11.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.424+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38006250 con 0x7f3f480b4070 2023-10-19T10:38:11.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.424+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f480c7db0 con 0x7f3f480b4070 2023-10-19T10:38:11.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.425+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f480c8e80 con 0x7f3f480b4070 2023-10-19T10:38:11.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.425+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3f380063b0 con 0x7f3f480b4070 2023-10-19T10:38:11.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.426+0000 7f3f2ffff640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f3406a050 0x7f3f3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.426+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3f380887e0 con 0x7f3f480b4070 2023-10-19T10:38:11.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.426+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f3406a050 0x7f3f3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.426+0000 7f3f46ffd640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f3406a050 0x7f3f3406c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c036000 tx=0x7f3f3c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.430+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3f480c8e80 con 0x7f3f480b4070 2023-10-19T10:38:11.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.606+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f480c8e80 con 0x7f3f480b4070 2023-10-19T10:38:11.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.606+0000 7f3f2ffff640 1 -- 172.21.15.50:0/2525179734 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f480c8e80 con 0x7f3f480b4070 2023-10-19T10:38:11.609 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:38:11.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f3406a050 msgr2=0x7f3f3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f3406a050 0x7f3f3406c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c036000 tx=0x7f3f3c03c040 comp rx=0 tx=0).stop 2023-10-19T10:38:11.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 msgr2=0x7f3f48155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 0x7f3f48155c20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c7c0 tx=0x7f3f3800cc80 comp rx=0 tx=0).stop 2023-10-19T10:38:11.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 shutdown_connections 2023-10-19T10:38:11.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f3406a050 0x7f3f3406c510 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3f480c7920 0x7f3f48145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4815c960 0x7f3f480c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 --2- 172.21.15.50:0/2525179734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f480b4070 0x7f3f48155c20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 >> 172.21.15.50:0/2525179734 conn(0x7f3f4809b970 msgr2=0x7f3f4806bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:11.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 shutdown_connections 2023-10-19T10:38:11.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.608+0000 7f3f4deee640 1 -- 172.21.15.50:0/2525179734 wait complete. 2023-10-19T10:38:11.621 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:11.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.728+0000 7f049ae58640 1 Processor -- start 2023-10-19T10:38:11.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.728+0000 7f049ae58640 1 -- start start 2023-10-19T10:38:11.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f049ae58640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f049415a650 0x7f049415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f049ae58640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f04940c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f049ae58640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049415ef60 con 0x7f049406e040 2023-10-19T10:38:11.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f049ae58640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049415aa30 con 0x7f049415a650 2023-10-19T10:38:11.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f049ae58640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049406e5c0 con 0x7f049406dbe0 2023-10-19T10:38:11.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0498bf4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f049415a650 0x7f049415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f04993f5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f049406e040 0x7f04940b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39378/0 (socket says 172.21.15.50:39378) 2023-10-19T10:38:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f04993f5640 1 -- 172.21.15.50:0/3208565996 learned_addr learned my addr 172.21.15.50:0/3208565996 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0493fff640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f04940c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0498bf4640 1 -- 172.21.15.50:0/3208565996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 msgr2=0x7f04940c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:11.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0498bf4640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f04940c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0498bf4640 1 -- 172.21.15.50:0/3208565996 >> v1:172.21.15.50:6789/0 conn(0x7f049406e040 legacy=0x7f04940b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0498bf4640 1 -- 172.21.15.50:0/3208565996 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049406e8a0 con 0x7f049415a650 2023-10-19T10:38:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.729+0000 7f0493fff640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f04940c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.730+0000 7f0498bf4640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049415a650 0x7f049415ea20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f04880097a0 tx=0x7f048802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5ae77e6cbf10f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.730+0000 7f04937fe640 1 -- 172.21.15.50:0/3208565996 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488031780 con 0x7f049415a650 2023-10-19T10:38:11.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.730+0000 7f04937fe640 1 -- 172.21.15.50:0/3208565996 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04880318e0 con 0x7f049415a650 2023-10-19T10:38:11.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.730+0000 7f04937fe640 1 -- 172.21.15.50:0/3208565996 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488031c70 con 0x7f049415a650 2023-10-19T10:38:11.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.730+0000 7f049ae58640 1 -- 172.21.15.50:0/3208565996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049415a650 msgr2=0x7f049415ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.730+0000 7f049ae58640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049415a650 0x7f049415ea20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f04880097a0 tx=0x7f048802b820 comp rx=0 tx=0).stop 2023-10-19T10:38:11.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.731+0000 7f049ae58640 1 -- 172.21.15.50:0/3208565996 shutdown_connections 2023-10-19T10:38:11.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.731+0000 7f049ae58640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f04940c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.731+0000 7f049ae58640 1 --2- 172.21.15.50:0/3208565996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049415a650 0x7f049415ea20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.731+0000 7f049ae58640 1 -- 172.21.15.50:0/3208565996 >> 172.21.15.50:0/3208565996 conn(0x7f04940cf170 msgr2=0x7f04940cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.731+0000 7f049ae58640 1 -- 172.21.15.50:0/3208565996 shutdown_connections 2023-10-19T10:38:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.731+0000 7f049ae58640 1 -- 172.21.15.50:0/3208565996 wait complete. 2023-10-19T10:38:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.732+0000 7f049ae58640 1 Processor -- start 2023-10-19T10:38:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.732+0000 7f049ae58640 1 -- start start 2023-10-19T10:38:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f049414e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 0x7f04940c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f049415a650 0x7f04940ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cded0 con 0x7f049406e040 2023-10-19T10:38:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cd2c0 con 0x7f049415a650 2023-10-19T10:38:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cd5a0 con 0x7f049406dbe0 2023-10-19T10:38:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 0x7f04940c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 0x7f04940c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50014/0 (socket says 172.21.15.50:50014) 2023-10-19T10:38:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 -- 172.21.15.50:0/30439590 learned_addr learned my addr 172.21.15.50:0/30439590 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0498bf4640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f049414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f04993f5640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f049415a650 0x7f04940ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 -- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 msgr2=0x7f049414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f049414e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 -- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f049415a650 msgr2=0x7f04940ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f049415a650 0x7f04940ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 -- 172.21.15.50:0/30439590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488009400 con 0x7f049406e040 2023-10-19T10:38:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f0493fff640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 0x7f04940c9d20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f048000c7f0 tx=0x7f048000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480007820 con 0x7f049406e040 2023-10-19T10:38:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480007980 con 0x7f049406e040 2023-10-19T10:38:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480007cf0 con 0x7f049406e040 2023-10-19T10:38:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.733+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04940cd880 con 0x7f049406e040 2023-10-19T10:38:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.734+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049414eab0 con 0x7f049406e040 2023-10-19T10:38:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.734+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04940ce080 con 0x7f049406e040 2023-10-19T10:38:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.735+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f04800113f0 con 0x7f049406e040 2023-10-19T10:38:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.735+0000 7f04917fa640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f047806a080 0x7f047806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.735+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0480088b50 con 0x7f049406e040 2023-10-19T10:38:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.736+0000 7f0498bf4640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f047806a080 0x7f047806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.736+0000 7f0498bf4640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f047806a080 0x7f047806c540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0488036000 tx=0x7f048803d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.740+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f04940ce080 con 0x7f049406e040 2023-10-19T10:38:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.915+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f049404e420 con 0x7f049406e040 2023-10-19T10:38:11.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.915+0000 7f04917fa640 1 -- 172.21.15.50:0/30439590 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0480053fc0 con 0x7f049406e040 2023-10-19T10:38:11.918 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:11.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f047806a080 msgr2=0x7f047806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f047806a080 0x7f047806c540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0488036000 tx=0x7f048803d040 comp rx=0 tx=0).stop 2023-10-19T10:38:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 msgr2=0x7f04940c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 0x7f04940c9d20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f048000c7f0 tx=0x7f048000ccb0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 shutdown_connections 2023-10-19T10:38:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f047806a080 0x7f047806c540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f049415a650 0x7f04940ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f049406e040 0x7f04940c9d20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 --2- 172.21.15.50:0/30439590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f049406dbe0 0x7f049414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 >> 172.21.15.50:0/30439590 conn(0x7f04940cf170 msgr2=0x7f049415d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 shutdown_connections 2023-10-19T10:38:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:11.917+0000 7f049ae58640 1 -- 172.21.15.50:0/30439590 wait complete. 2023-10-19T10:38:11.930 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:38:12.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.035+0000 7f0ce6302640 1 Processor -- start 2023-10-19T10:38:12.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.035+0000 7f0ce6302640 1 -- start start 2023-10-19T10:38:12.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.035+0000 7f0ce6302640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0ce00d0e30 0x7f0ce015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0ce6302640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce006aa30 con 0x7f0ce00cd920 2023-10-19T10:38:12.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0ce6302640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce006b2d0 con 0x7f0ce015c570 2023-10-19T10:38:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0ce6302640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce015ce90 con 0x7f0ce00d0e30 2023-10-19T10:38:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0ce489f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f0ce00cd920 0x7f0ce00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39388/0 (socket says 172.21.15.50:39388) 2023-10-19T10:38:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0ce489f640 1 -- 172.21.15.50:0/3107103502 learned_addr learned my addr 172.21.15.50:0/3107103502 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835827214 0 0) 0x7f0ce006aa30 con 0x7f0ce00cd920 2023-10-19T10:38:12.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdf7fe640 1 --2- 172.21.15.50:0/3107103502 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00d0e30 0x7f0ce015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc8003520 con 0x7f0ce00cd920 2023-10-19T10:38:12.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428243177 0 0) 0x7f0cc8003520 con 0x7f0ce00cd920 2023-10-19T10:38:12.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00d0e30 msgr2=0x7f0ce015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 --2- 172.21.15.50:0/3107103502 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00d0e30 0x7f0ce015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 >> v1:172.21.15.183:6789/0 conn(0x7f0ce015c570 legacy=0x7f0ce006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00d1270 con 0x7f0ce00cd920 2023-10-19T10:38:12.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f0cd80019f0 con 0x7f0ce00cd920 2023-10-19T10:38:12.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 >> v1:172.21.15.50:6789/0 conn(0x7f0ce00cd920 legacy=0x7f0ce00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc8003970 0x7f0cc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cc8005180 0x7f0cc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce015ce90 con 0x7f0cc8005180 2023-10-19T10:38:12.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc800c120 con 0x7f0cc8003970 2023-10-19T10:38:12.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc800c2e0 con 0x7f0cc80047c0 2023-10-19T10:38:12.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.036+0000 7f0ce489f640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cc8005180 0x7f0cc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0cdffff640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc8003970 0x7f0cc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0cdf7fe640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0ce489f640 1 -- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80047c0 msgr2=0x7f0cc8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0ce489f640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0ce489f640 1 -- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc8003970 msgr2=0x7f0cc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0ce489f640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc8003970 0x7f0cc8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0ce489f640 1 -- 172.21.15.50:0/3107103502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8003520 con 0x7f0cc8005180 2023-10-19T10:38:12.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.037+0000 7f0ce489f640 1 -- 172.21.15.50:0/3107103502 reap_dead start 2023-10-19T10:38:12.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce489f640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cc8005180 0x7f0cc800b980 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800a830 tx=0x7f0cc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a404fdb57747ec66 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80072d0 con 0x7f0cc8005180 2023-10-19T10:38:12.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd802f070 con 0x7f0cc8005180 2023-10-19T10:38:12.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0cdeffd640 1 -- 172.21.15.50:0/3107103502 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80076b0 con 0x7f0cc8005180 2023-10-19T10:38:12.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 -- 172.21.15.50:0/3107103502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cc8005180 msgr2=0x7f0cc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cc8005180 0x7f0cc800b980 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800a830 tx=0x7f0cc80057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 -- 172.21.15.50:0/3107103502 shutdown_connections 2023-10-19T10:38:12.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 --2- 172.21.15.50:0/3107103502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cc8005180 0x7f0cc800b980 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 -- 172.21.15.50:0/3107103502 >> 172.21.15.50:0/3107103502 conn(0x7f0ce0089ff0 msgr2=0x7f0ce009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:12.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 -- 172.21.15.50:0/3107103502 shutdown_connections 2023-10-19T10:38:12.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.038+0000 7f0ce6302640 1 -- 172.21.15.50:0/3107103502 wait complete. 2023-10-19T10:38:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.039+0000 7f0ce6302640 1 Processor -- start 2023-10-19T10:38:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 -- start start 2023-10-19T10:38:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc8005180 0x7f0ce0145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 0x7f0ce00c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00c8f70 0x7f0ce0147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce015e020 con 0x7f0ce0158730 2023-10-19T10:38:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0145fc0 con 0x7f0ce00c8f70 2023-10-19T10:38:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0ce6302640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c9410 con 0x7f0cc8005180 2023-10-19T10:38:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0cdf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 0x7f0ce00c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0cdf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 0x7f0ce00c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50024/0 (socket says 172.21.15.50:50024) 2023-10-19T10:38:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.040+0000 7f0cdf7fe640 1 -- 172.21.15.50:0/1502389824 learned_addr learned my addr 172.21.15.50:0/1502389824 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdffff640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc8005180 0x7f0ce0145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0ce489f640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00c8f70 0x7f0ce0147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdf7fe640 1 -- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc8005180 msgr2=0x7f0ce0145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdf7fe640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc8005180 0x7f0ce0145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdf7fe640 1 -- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00c8f70 msgr2=0x7f0ce0147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdf7fe640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00c8f70 0x7f0ce0147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdf7fe640 1 -- 172.21.15.50:0/1502389824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce015e270 con 0x7f0ce0158730 2023-10-19T10:38:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdf7fe640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 0x7f0ce00c8a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4011910 tx=0x7f0cd4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400faf0 con 0x7f0ce0158730 2023-10-19T10:38:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400fc50 con 0x7f0ce0158730 2023-10-19T10:38:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce015e550 con 0x7f0ce0158730 2023-10-19T10:38:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4014440 con 0x7f0ce0158730 2023-10-19T10:38:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.041+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0148820 con 0x7f0ce0158730 2023-10-19T10:38:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.042+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce0073990 con 0x7f0ce0158730 2023-10-19T10:38:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.043+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0cd40145a0 con 0x7f0ce0158730 2023-10-19T10:38:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.043+0000 7f0cdcff9640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ccc06a050 0x7f0ccc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.043+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0cd40890c0 con 0x7f0ce0158730 2023-10-19T10:38:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.043+0000 7f0cdffff640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ccc06a050 0x7f0ccc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.044+0000 7f0cdffff640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ccc06a050 0x7f0ccc06c510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000c00 tx=0x7f0cd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.047+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0cd4054480 con 0x7f0ce0158730 2023-10-19T10:38:12.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.234+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ce006ac50 con 0x7f0ce0158730 2023-10-19T10:38:12.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.235+0000 7f0cdcff9640 1 -- 172.21.15.50:0/1502389824 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0cd4057aa0 con 0x7f0ce0158730 2023-10-19T10:38:12.238 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2023-10-19T10:38:12.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ccc06a050 msgr2=0x7f0ccc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ccc06a050 0x7f0ccc06c510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000c00 tx=0x7f0cd0009040 comp rx=0 tx=0).stop 2023-10-19T10:38:12.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 msgr2=0x7f0ce00c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 0x7f0ce00c8a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4011910 tx=0x7f0cd4011dd0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 shutdown_connections 2023-10-19T10:38:12.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0ccc06a050 0x7f0ccc06c510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00c8f70 0x7f0ce0147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ce0158730 0x7f0ce00c8a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 --2- 172.21.15.50:0/1502389824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc8005180 0x7f0ce0145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.236+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 >> 172.21.15.50:0/1502389824 conn(0x7f0ce0089ff0 msgr2=0x7f0ce00d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:12.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.237+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 shutdown_connections 2023-10-19T10:38:12.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.237+0000 7f0ce6302640 1 -- 172.21.15.50:0/1502389824 wait complete. 2023-10-19T10:38:12.250 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi183.front.sepia.ceph.com... 2023-10-19T10:38:12.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:12.250 DEBUG:teuthology.orchestra.run.smithi183:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-19T10:38:12.283 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:12.284 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:38:12.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:12.285 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-19T10:38:12.341 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-19T10:38:12.342 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:12.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.445+0000 7f3300a99640 1 Processor -- start 2023-10-19T10:38:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.445+0000 7f3300a99640 1 -- start start 2023-10-19T10:38:12.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.446+0000 7f3300a99640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f32fc15c570 0x7f32fc0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.446+0000 7f3300a99640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32fc0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.446+0000 7f3300a99640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc06aa30 con 0x7f32fc15c950 2023-10-19T10:38:12.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.446+0000 7f3300a99640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32fc06b2d0 con 0x7f32fc15c570 2023-10-19T10:38:12.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.446+0000 7f3300a99640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0ce090 con 0x7f32fc0cd920 2023-10-19T10:38:12.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f9d9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f32fc15c950 0x7f32fc06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39400/0 (socket says 172.21.15.50:39400) 2023-10-19T10:38:12.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f9d9b640 1 -- 172.21.15.50:0/3286009177 learned_addr learned my addr 172.21.15.50:0/3286009177 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:12.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3286009177 >> v2:172.21.15.183:3300/0 conn(0x7f32fc15c570 0x7f32fc0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32fc0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063725501 0 0) 0x7f32fc06aa30 con 0x7f32fc15c950 2023-10-19T10:38:12.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e8003520 con 0x7f32fc15c950 2023-10-19T10:38:12.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 777746058 0 0) 0x7f32e8003520 con 0x7f32fc15c950 2023-10-19T10:38:12.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 msgr2=0x7f32fc0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32fc0d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 >> v2:172.21.15.183:3300/0 conn(0x7f32fc15c570 msgr2=0x7f32fc0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 --2- 172.21.15.50:0/3286009177 >> v2:172.21.15.183:3300/0 conn(0x7f32fc15c570 0x7f32fc0d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc06ac80 con 0x7f32fc15c950 2023-10-19T10:38:12.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f32e4001670 con 0x7f32fc15c950 2023-10-19T10:38:12.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 >> v1:172.21.15.50:6789/0 conn(0x7f32fc15c950 legacy=0x7f32fc06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32e8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8003ef0 0x7f32e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32e8004d00 0x7f32e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc06b2d0 con 0x7f32e8004d00 2023-10-19T10:38:12.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0ce090 con 0x7f32e8003ef0 2023-10-19T10:38:12.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e800b9d0 con 0x7f32fc0cd920 2023-10-19T10:38:12.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3286009177 >> v2:172.21.15.183:3300/0 conn(0x7f32fc15c570 0x7f32fc0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:12.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32e8004d00 0x7f32e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32e8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.447+0000 7f32f9d9b640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8003ef0 0x7f32e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fad9d640 1 -- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 msgr2=0x7f32e8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32e8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fad9d640 1 -- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8003ef0 msgr2=0x7f32e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8003ef0 0x7f32e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fad9d640 1 -- 172.21.15.50:0/3286009177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8003520 con 0x7f32e8004d00 2023-10-19T10:38:12.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc0cd920 0x7f32e8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:12.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32f9d9b640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8003ef0 0x7f32e80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:12.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.448+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32e8004d00 0x7f32e8005180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e170 tx=0x7f32e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f06bd6bcb09992 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec02cbf0 con 0x7f32e8004d00 2023-10-19T10:38:12.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec00f040 con 0x7f32e8004d00 2023-10-19T10:38:12.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f32f959a640 1 -- 172.21.15.50:0/3286009177 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec034440 con 0x7f32e8004d00 2023-10-19T10:38:12.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f3300a99640 1 -- 172.21.15.50:0/3286009177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32e8004d00 msgr2=0x7f32e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f3300a99640 1 --2- 172.21.15.50:0/3286009177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32e8004d00 0x7f32e8005180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e170 tx=0x7f32e80057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f32fad9d640 1 -- 172.21.15.50:0/3286009177 reap_dead start 2023-10-19T10:38:12.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f3300a99640 1 -- 172.21.15.50:0/3286009177 shutdown_connections 2023-10-19T10:38:12.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f3300a99640 1 -- 172.21.15.50:0/3286009177 >> 172.21.15.50:0/3286009177 conn(0x7f32fc089ff0 msgr2=0x7f32fc09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:12.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f3300a99640 1 -- 172.21.15.50:0/3286009177 shutdown_connections 2023-10-19T10:38:12.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.449+0000 7f3300a99640 1 -- 172.21.15.50:0/3286009177 wait complete. 2023-10-19T10:38:12.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 Processor -- start 2023-10-19T10:38:12.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 -- start start 2023-10-19T10:38:12.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8004d00 0x7f32fc0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc1581c0 0x7f32fc0afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 0x7f32fc0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0b0610 con 0x7f32fc0b0190 2023-10-19T10:38:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0c93e0 con 0x7f32e8004d00 2023-10-19T10:38:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f3300a99640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1e8110 con 0x7f32fc1581c0 2023-10-19T10:38:12.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.451+0000 7f32fad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 0x7f32fc0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 0x7f32fc0c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50032/0 (socket says 172.21.15.50:50032) 2023-10-19T10:38:12.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 -- 172.21.15.50:0/3412010359 learned_addr learned my addr 172.21.15.50:0/3412010359 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:12.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8004d00 0x7f32fc0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32f9d9b640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc1581c0 0x7f32fc0afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 -- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc1581c0 msgr2=0x7f32fc0afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc1581c0 0x7f32fc0afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 -- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8004d00 msgr2=0x7f32fc0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8004d00 0x7f32fc0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 -- 172.21.15.50:0/3412010359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc158a90 con 0x7f32fc0b0190 2023-10-19T10:38:12.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32fad9d640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 0x7f32fc0c7380 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32ec037040 tx=0x7f32ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec003990 con 0x7f32fc0b0190 2023-10-19T10:38:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec003af0 con 0x7f32fc0b0190 2023-10-19T10:38:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec003ea0 con 0x7f32fc0b0190 2023-10-19T10:38:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc158d70 con 0x7f32fc0b0190 2023-10-19T10:38:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.452+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc158640 con 0x7f32fc0b0190 2023-10-19T10:38:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.453+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc1588f0 con 0x7f32fc0b0190 2023-10-19T10:38:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.454+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f32ec034940 con 0x7f32fc0b0190 2023-10-19T10:38:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.454+0000 7f32f2ffd640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f32dc06a050 0x7f32dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.454+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f32ec0aa5c0 con 0x7f32fc0b0190 2023-10-19T10:38:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.454+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f32dc06a050 0x7f32dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.455+0000 7f32fa59c640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f32dc06a050 0x7f32dc06c510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009c90 tx=0x7f32f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.459+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f32fc1588f0 con 0x7f32fc0b0190 2023-10-19T10:38:12.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.669+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32fc1588f0 con 0x7f32fc0b0190 2023-10-19T10:38:12.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.670+0000 7f32f2ffd640 1 -- 172.21.15.50:0/3412010359 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7f32fc1588f0 con 0x7f32fc0b0190 2023-10-19T10:38:12.673 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:12.673 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:38:12.674 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:38:12.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f32dc06a050 msgr2=0x7f32dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f32dc06a050 0x7f32dc06c510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009c90 tx=0x7f32f4008040 comp rx=0 tx=0).stop 2023-10-19T10:38:12.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 msgr2=0x7f32fc0c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 0x7f32fc0c7380 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32ec037040 tx=0x7f32ec009750 comp rx=0 tx=0).stop 2023-10-19T10:38:12.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 shutdown_connections 2023-10-19T10:38:12.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f32dc06a050 0x7f32dc06c510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32fc0b0190 0x7f32fc0c7380 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32fc1581c0 0x7f32fc0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 --2- 172.21.15.50:0/3412010359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8004d00 0x7f32fc0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 >> 172.21.15.50:0/3412010359 conn(0x7f32fc089ff0 msgr2=0x7f32fc15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:12.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 shutdown_connections 2023-10-19T10:38:12.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.672+0000 7f3300a99640 1 -- 172.21.15.50:0/3412010359 wait complete. 2023-10-19T10:38:12.686 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:12.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.789+0000 7f37b9ef4640 1 Processor -- start 2023-10-19T10:38:12.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.789+0000 7f37b9ef4640 1 -- start start 2023-10-19T10:38:12.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b9ef4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37b415c570 0x7f37b40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b9ef4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b40d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b9ef4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b406aa30 con 0x7f37b415c950 2023-10-19T10:38:12.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b9ef4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b406b2d0 con 0x7f37b415c570 2023-10-19T10:38:12.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b9ef4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40ce090 con 0x7f37b40cd920 2023-10-19T10:38:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37b415c570 0x7f37b40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b2ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f37b415c950 0x7f37b406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39406/0 (socket says 172.21.15.50:39406) 2023-10-19T10:38:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b2ffd640 1 -- 172.21.15.50:0/2685511500 learned_addr learned my addr 172.21.15.50:0/2685511500 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:12.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b3fff640 1 --2- 172.21.15.50:0/2685511500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b40d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b37fe640 1 -- 172.21.15.50:0/2685511500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 msgr2=0x7f37b40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:12.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b37fe640 1 --2- 172.21.15.50:0/2685511500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b40d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b37fe640 1 -- 172.21.15.50:0/2685511500 >> v1:172.21.15.50:6789/0 conn(0x7f37b415c950 legacy=0x7f37b406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b37fe640 1 -- 172.21.15.50:0/2685511500 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b406ac80 con 0x7f37b415c570 2023-10-19T10:38:12.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.790+0000 7f37b27fc640 1 -- 172.21.15.50:0/2685511500 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894799745 0 0) 0x7f37b406aa30 con 0x7f37b415c950 2023-10-19T10:38:12.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.791+0000 7f37b37fe640 1 --2- 172.21.15.50:0/2685511500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c570 0x7f37b40d0e30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009790 tx=0x7f37a402b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25815477859c3ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.791+0000 7f37b27fc640 1 -- 172.21.15.50:0/2685511500 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4031770 con 0x7f37b415c570 2023-10-19T10:38:12.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.791+0000 7f37b27fc640 1 -- 172.21.15.50:0/2685511500 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a40318d0 con 0x7f37b415c570 2023-10-19T10:38:12.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.791+0000 7f37b27fc640 1 -- 172.21.15.50:0/2685511500 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4031c60 con 0x7f37b415c570 2023-10-19T10:38:12.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 -- 172.21.15.50:0/2685511500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c570 msgr2=0x7f37b40d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/2685511500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c570 0x7f37b40d0e30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009790 tx=0x7f37a402b810 comp rx=0 tx=0).stop 2023-10-19T10:38:12.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 -- 172.21.15.50:0/2685511500 shutdown_connections 2023-10-19T10:38:12.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/2685511500 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b40d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/2685511500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c570 0x7f37b40d0e30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 -- 172.21.15.50:0/2685511500 >> 172.21.15.50:0/2685511500 conn(0x7f37b4089ff0 msgr2=0x7f37b409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:12.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 -- 172.21.15.50:0/2685511500 shutdown_connections 2023-10-19T10:38:12.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.792+0000 7f37b9ef4640 1 -- 172.21.15.50:0/2685511500 wait complete. 2023-10-19T10:38:12.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.793+0000 7f37b9ef4640 1 Processor -- start 2023-10-19T10:38:12.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.793+0000 7f37b9ef4640 1 -- start start 2023-10-19T10:38:12.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b9ef4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b9ef4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b415c570 0x7f37b40c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b9ef4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 0x7f37b40c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b9ef4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40c6d10 con 0x7f37b415c950 2023-10-19T10:38:12.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b9ef4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40c91b0 con 0x7f37b415c570 2023-10-19T10:38:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b9ef4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b415ac70 con 0x7f37b40cd920 2023-10-19T10:38:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 0x7f37b40c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 0x7f37b40c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50056/0 (socket says 172.21.15.50:50056) 2023-10-19T10:38:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 -- 172.21.15.50:0/4089471217 learned_addr learned my addr 172.21.15.50:0/4089471217 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:12.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b37fe640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 -- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 msgr2=0x7f37b415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b2ffd640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b415c570 0x7f37b40c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 -- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b415c570 msgr2=0x7f37b40c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b415c570 0x7f37b40c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 -- 172.21.15.50:0/4089471217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40093a0 con 0x7f37b415c950 2023-10-19T10:38:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b2ffd640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b415c570 0x7f37b40c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.794+0000 7f37b3fff640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 0x7f37b40c86a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37ac007d20 tx=0x7f37ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.795+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac014420 con 0x7f37b415c950 2023-10-19T10:38:12.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.795+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac0145a0 con 0x7f37b415c950 2023-10-19T10:38:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.795+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b40c6370 con 0x7f37b415c950 2023-10-19T10:38:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.795+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac014950 con 0x7f37b415c950 2023-10-19T10:38:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.795+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b40c6930 con 0x7f37b415c950 2023-10-19T10:38:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.796+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b40c6e50 con 0x7f37b415c950 2023-10-19T10:38:12.810 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.796+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f37ac014ab0 con 0x7f37b415c950 2023-10-19T10:38:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.796+0000 7f379bfff640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a006a050 0x7f37a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.797+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f37ac089230 con 0x7f37b415c950 2023-10-19T10:38:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.797+0000 7f37b37fe640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a006a050 0x7f37a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.797+0000 7f37b37fe640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a006a050 0x7f37a006c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f37a4036000 tx=0x7f37a403d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:12.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.801+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f37b40c6e50 con 0x7f37b415c950 2023-10-19T10:38:12.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.976+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37b40c6e50 con 0x7f37b415c950 2023-10-19T10:38:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.978+0000 7f379bfff640 1 -- 172.21.15.50:0/4089471217 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f37b40c6e50 con 0x7f37b415c950 2023-10-19T10:38:12.980 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:12.981 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:12.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a006a050 msgr2=0x7f37a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a006a050 0x7f37a006c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f37a4036000 tx=0x7f37a403d040 comp rx=0 tx=0).stop 2023-10-19T10:38:12.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 msgr2=0x7f37b40c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:12.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 0x7f37b40c86a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37ac007d20 tx=0x7f37ac00e4b0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 shutdown_connections 2023-10-19T10:38:12.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a006a050 0x7f37a006c510 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37b415c950 0x7f37b40c86a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b415c570 0x7f37b40c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 --2- 172.21.15.50:0/4089471217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37b40cd920 0x7f37b415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:12.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 >> 172.21.15.50:0/4089471217 conn(0x7f37b4089ff0 msgr2=0x7f37b4072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:12.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.979+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 shutdown_connections 2023-10-19T10:38:12.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:12.980+0000 7f37b9ef4640 1 -- 172.21.15.50:0/4089471217 wait complete. 2023-10-19T10:38:12.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:36:53.037250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:36:53.634564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-19T10:36:57.357722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-19T10:38:12.994 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:13.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.101+0000 7f4082502640 1 Processor -- start 2023-10-19T10:38:13.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.101+0000 7f4082502640 1 -- start start 2023-10-19T10:38:13.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f4082502640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f407c0ce8a0 0x7f407c0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f4082502640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 0x7f407c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f4082502640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0cec80 con 0x7f407c15d060 2023-10-19T10:38:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f4082502640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407c0d0b80 con 0x7f407c0ce8a0 2023-10-19T10:38:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f4082502640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c15dd90 con 0x7f407c0d1010 2023-10-19T10:38:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 0x7f407c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 0x7f407c0cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:48824/0 (socket says 172.21.15.50:48824) 2023-10-19T10:38:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 -- 172.21.15.50:0/4092886019 learned_addr learned my addr 172.21.15.50:0/4092886019 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407bfff640 1 --2- 172.21.15.50:0/4092886019 >> v2:172.21.15.183:3300/0 conn(0x7f407c0ce8a0 0x7f407c0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 -- 172.21.15.50:0/4092886019 >> v2:172.21.15.183:3300/0 conn(0x7f407c0ce8a0 msgr2=0x7f407c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 --2- 172.21.15.50:0/4092886019 >> v2:172.21.15.183:3300/0 conn(0x7f407c0ce8a0 0x7f407c0cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 -- 172.21.15.50:0/4092886019 >> v1:172.21.15.183:6789/0 conn(0x7f407c15d060 legacy=0x7f407c15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.102+0000 7f407b7fe640 1 -- 172.21.15.50:0/4092886019 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0d0250 con 0x7f407c0d1010 2023-10-19T10:38:13.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.103+0000 7f407b7fe640 1 --2- 172.21.15.50:0/4092886019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 0x7f407c0cd3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4070009870 tx=0x7f407002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52edba649f1aaa36 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.103+0000 7f407affd640 1 -- 172.21.15.50:0/4092886019 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070031640 con 0x7f407c0d1010 2023-10-19T10:38:13.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.103+0000 7f407affd640 1 -- 172.21.15.50:0/4092886019 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40700317a0 con 0x7f407c0d1010 2023-10-19T10:38:13.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.103+0000 7f407affd640 1 -- 172.21.15.50:0/4092886019 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070031b30 con 0x7f407c0d1010 2023-10-19T10:38:13.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 -- 172.21.15.50:0/4092886019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 msgr2=0x7f407c0cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 --2- 172.21.15.50:0/4092886019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 0x7f407c0cd3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4070009870 tx=0x7f407002b8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 -- 172.21.15.50:0/4092886019 shutdown_connections 2023-10-19T10:38:13.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 --2- 172.21.15.50:0/4092886019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0d1010 0x7f407c0cd3d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 --2- 172.21.15.50:0/4092886019 >> v2:172.21.15.183:3300/0 conn(0x7f407c0ce8a0 0x7f407c0cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 -- 172.21.15.50:0/4092886019 >> 172.21.15.50:0/4092886019 conn(0x7f407c069800 msgr2=0x7f407c069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:13.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 -- 172.21.15.50:0/4092886019 shutdown_connections 2023-10-19T10:38:13.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.104+0000 7f4082502640 1 -- 172.21.15.50:0/4092886019 wait complete. 2023-10-19T10:38:13.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.105+0000 7f4082502640 1 Processor -- start 2023-10-19T10:38:13.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.105+0000 7f4082502640 1 -- start start 2023-10-19T10:38:13.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4082502640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0ce8a0 0x7f407c1f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4082502640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f407c0d1010 0x7f407c15ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4082502640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 0x7f407c160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4082502640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0c4fa0 con 0x7f407c15d060 2023-10-19T10:38:13.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4082502640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c1f66f0 con 0x7f407c0d1010 2023-10-19T10:38:13.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4082502640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0c4390 con 0x7f407c0ce8a0 2023-10-19T10:38:13.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 0x7f407c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 0x7f407c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50064/0 (socket says 172.21.15.50:50064) 2023-10-19T10:38:13.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 -- 172.21.15.50:0/1849510304 learned_addr learned my addr 172.21.15.50:0/1849510304 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:13.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f407bfff640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0ce8a0 0x7f407c1f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f407b7fe640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f407c0d1010 0x7f407c15ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 -- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0ce8a0 msgr2=0x7f407c1f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0ce8a0 0x7f407c1f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 -- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f407c0d1010 msgr2=0x7f407c15ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f407c0d1010 0x7f407c15ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f4080a9f640 1 -- 172.21.15.50:0/1849510304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40700094d0 con 0x7f407c15d060 2023-10-19T10:38:13.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.106+0000 7f407b7fe640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f407c0d1010 0x7f407c15ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:13.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.107+0000 7f4080a9f640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 0x7f407c160980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f407400c6b0 tx=0x7f407400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.107+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074003b70 con 0x7f407c15d060 2023-10-19T10:38:13.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.107+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074003cf0 con 0x7f407c15d060 2023-10-19T10:38:13.120 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.107+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c0c4670 con 0x7f407c15d060 2023-10-19T10:38:13.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.107+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074003ea0 con 0x7f407c15d060 2023-10-19T10:38:13.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.107+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c0c4c30 con 0x7f407c15d060 2023-10-19T10:38:13.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.108+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c0c58b0 con 0x7f407c15d060 2023-10-19T10:38:13.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.108+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4074005310 con 0x7f407c15d060 2023-10-19T10:38:13.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.109+0000 7f4078ff9640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f406806a080 0x7f406806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.109+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4074088570 con 0x7f407c15d060 2023-10-19T10:38:13.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.109+0000 7f407bfff640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f406806a080 0x7f406806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.109+0000 7f407bfff640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f406806a080 0x7f406806c540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f406c009b40 tx=0x7f406c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.113+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f407c0c58b0 con 0x7f407c15d060 2023-10-19T10:38:13.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.288+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f407c0c58b0 con 0x7f407c15d060 2023-10-19T10:38:13.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.289+0000 7f4078ff9640 1 -- 172.21.15.50:0/1849510304 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f407c0c58b0 con 0x7f407c15d060 2023-10-19T10:38:13.292 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:13.293 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.290+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f406806a080 msgr2=0x7f406806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.290+0000 7f4082502640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f406806a080 0x7f406806c540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f406c009b40 tx=0x7f406c009190 comp rx=0 tx=0).stop 2023-10-19T10:38:13.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 msgr2=0x7f407c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 0x7f407c160980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f407400c6b0 tx=0x7f407400cb70 comp rx=0 tx=0).stop 2023-10-19T10:38:13.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 shutdown_connections 2023-10-19T10:38:13.294 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f406806a080 0x7f406806c540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f407c15d060 0x7f407c160980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f407c0d1010 0x7f407c15ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 --2- 172.21.15.50:0/1849510304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f407c0ce8a0 0x7f407c1f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 >> 172.21.15.50:0/1849510304 conn(0x7f407c069800 msgr2=0x7f407c0b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:13.295 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 shutdown_connections 2023-10-19T10:38:13.296 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.291+0000 7f4082502640 1 -- 172.21.15.50:0/1849510304 wait complete. 2023-10-19T10:38:13.304 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:13.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 Processor -- start 2023-10-19T10:38:13.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 -- start start 2023-10-19T10:38:13.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d3415c6f0 0x7f0d340ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0d340afab0 0x7f0d340ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3406f2a0 con 0x7f0d340b1f60 2023-10-19T10:38:13.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3406fe20 con 0x7f0d340afab0 2023-10-19T10:38:13.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.411+0000 7f0d3b163640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340702b0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d38eff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d3415c6f0 0x7f0d340ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d39700640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0d340afab0 0x7f0d340ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d33fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0d340b1f60 0x7f0d34160260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:37680/0 (socket says 172.21.15.50:37680) 2023-10-19T10:38:13.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d33fff640 1 -- 172.21.15.50:0/1364161788 learned_addr learned my addr 172.21.15.50:0/1364161788 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:13.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d38eff640 1 -- 172.21.15.50:0/1364161788 >> v2:172.21.15.183:3300/0 conn(0x7f0d340afab0 msgr2=0x7f0d340ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:13.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d38eff640 1 --2- 172.21.15.50:0/1364161788 >> v2:172.21.15.183:3300/0 conn(0x7f0d340afab0 0x7f0d340ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d337fe640 1 -- 172.21.15.50:0/1364161788 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410802637 0 0) 0x7f0d3406f2a0 con 0x7f0d340b1f60 2023-10-19T10:38:13.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d38eff640 1 -- 172.21.15.50:0/1364161788 >> v1:172.21.15.183:6789/0 conn(0x7f0d340b1f60 legacy=0x7f0d34160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.412+0000 7f0d38eff640 1 -- 172.21.15.50:0/1364161788 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3406f4f0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.413+0000 7f0d38eff640 1 --2- 172.21.15.50:0/1364161788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d3415c6f0 0x7f0d340ce020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d280097a0 tx=0x7f0d2802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79019e294e68b8dd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.413+0000 7f0d337fe640 1 -- 172.21.15.50:0/1364161788 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28031830 con 0x7f0d3415c6f0 2023-10-19T10:38:13.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.413+0000 7f0d337fe640 1 -- 172.21.15.50:0/1364161788 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28031990 con 0x7f0d3415c6f0 2023-10-19T10:38:13.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.413+0000 7f0d337fe640 1 -- 172.21.15.50:0/1364161788 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28031d20 con 0x7f0d3415c6f0 2023-10-19T10:38:13.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.413+0000 7f0d3b163640 1 -- 172.21.15.50:0/1364161788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d3415c6f0 msgr2=0x7f0d340ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.413+0000 7f0d3b163640 1 --2- 172.21.15.50:0/1364161788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d3415c6f0 0x7f0d340ce020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d280097a0 tx=0x7f0d2802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.414+0000 7f0d3b163640 1 -- 172.21.15.50:0/1364161788 shutdown_connections 2023-10-19T10:38:13.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.414+0000 7f0d3b163640 1 --2- 172.21.15.50:0/1364161788 >> v2:172.21.15.183:3300/0 conn(0x7f0d340afab0 0x7f0d340ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.414+0000 7f0d3b163640 1 --2- 172.21.15.50:0/1364161788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d3415c6f0 0x7f0d340ce020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.414+0000 7f0d3b163640 1 -- 172.21.15.50:0/1364161788 >> 172.21.15.50:0/1364161788 conn(0x7f0d340ccf60 msgr2=0x7f0d340cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:13.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.414+0000 7f0d3b163640 1 -- 172.21.15.50:0/1364161788 shutdown_connections 2023-10-19T10:38:13.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.414+0000 7f0d3b163640 1 -- 172.21.15.50:0/1364161788 wait complete. 2023-10-19T10:38:13.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.415+0000 7f0d3b163640 1 Processor -- start 2023-10-19T10:38:13.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.415+0000 7f0d3b163640 1 -- start start 2023-10-19T10:38:13.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d340afab0 0x7f0d340c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 0x7f0d340c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d340c8ab0 0x7f0d340c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340b4430 con 0x7f0d3415c6f0 2023-10-19T10:38:13.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340b3820 con 0x7f0d340afab0 2023-10-19T10:38:13.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340b3b00 con 0x7f0d340c8ab0 2023-10-19T10:38:13.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 0x7f0d340c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 0x7f0d340c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50078/0 (socket says 172.21.15.50:50078) 2023-10-19T10:38:13.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 -- 172.21.15.50:0/3944244505 learned_addr learned my addr 172.21.15.50:0/3944244505 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:13.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d38eff640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d340afab0 0x7f0d340c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d39700640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d340c8ab0 0x7f0d340c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 -- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d340c8ab0 msgr2=0x7f0d340c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d340c8ab0 0x7f0d340c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 -- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d340afab0 msgr2=0x7f0d340c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d340afab0 0x7f0d340c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 -- 172.21.15.50:0/3944244505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009400 con 0x7f0d3415c6f0 2023-10-19T10:38:13.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d33fff640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 0x7f0d340c8570 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c430 tx=0x7f0d2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d200074e0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d20007660 con 0x7f0d3415c6f0 2023-10-19T10:38:13.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20004a20 con 0x7f0d3415c6f0 2023-10-19T10:38:13.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.416+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340b3de0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.417+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d340c6cc0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.418+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340b45e0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.418+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0d20004b80 con 0x7f0d3415c6f0 2023-10-19T10:38:13.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.418+0000 7f0d317fa640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d1806a050 0x7f0d1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.419+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0d20088580 con 0x7f0d3415c6f0 2023-10-19T10:38:13.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.419+0000 7f0d38eff640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d1806a050 0x7f0d1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.419+0000 7f0d38eff640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d1806a050 0x7f0d1806c510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d28036000 tx=0x7f0d2803e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.423+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0d340b45e0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.596+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d3404e420 con 0x7f0d3415c6f0 2023-10-19T10:38:13.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.597+0000 7f0d317fa640 1 -- 172.21.15.50:0/3944244505 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d200539f0 con 0x7f0d3415c6f0 2023-10-19T10:38:13.601 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:13.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d1806a050 msgr2=0x7f0d1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d1806a050 0x7f0d1806c510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d28036000 tx=0x7f0d2803e040 comp rx=0 tx=0).stop 2023-10-19T10:38:13.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 msgr2=0x7f0d340c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 0x7f0d340c8570 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c430 tx=0x7f0d2000c8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 shutdown_connections 2023-10-19T10:38:13.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d1806a050 0x7f0d1806c510 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d340c8ab0 0x7f0d340c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d3415c6f0 0x7f0d340c8570 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 --2- 172.21.15.50:0/3944244505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d340afab0 0x7f0d340c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 >> 172.21.15.50:0/3944244505 conn(0x7f0d340ccf60 msgr2=0x7f0d34156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:13.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 shutdown_connections 2023-10-19T10:38:13.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.599+0000 7f0d3b163640 1 -- 172.21.15.50:0/3944244505 wait complete. 2023-10-19T10:38:13.613 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:38:13.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.718+0000 7feb3fa34640 1 Processor -- start 2023-10-19T10:38:13.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.719+0000 7feb3fa34640 1 -- start start 2023-10-19T10:38:13.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.719+0000 7feb3fa34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb380cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.719+0000 7feb3fa34640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb380cc0f0 con 0x7feb38158a10 2023-10-19T10:38:13.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.719+0000 7feb3fa34640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb380cd770 con 0x7feb38071130 2023-10-19T10:38:13.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.719+0000 7feb3fa34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb38158df0 con 0x7feb380cce20 2023-10-19T10:38:13.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3d7d0640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7feb38158a10 0x7feb38070540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39412/0 (socket says 172.21.15.50:39412) 2023-10-19T10:38:13.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3d7d0640 1 -- 172.21.15.50:0/1643317971 learned_addr learned my addr 172.21.15.50:0/1643317971 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:13.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336495170 0 0) 0x7feb380cc0f0 con 0x7feb38158a10 2023-10-19T10:38:13.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb380cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb24003520 con 0x7feb38158a10 2023-10-19T10:38:13.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1237768432 0 0) 0x7feb24003520 con 0x7feb38158a10 2023-10-19T10:38:13.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 msgr2=0x7feb380cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:13.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb380cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 >> v1:172.21.15.183:6789/0 conn(0x7feb38071130 legacy=0x7feb3806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38070d50 con 0x7feb38158a10 2023-10-19T10:38:13.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7feb34001670 con 0x7feb38158a10 2023-10-19T10:38:13.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb380cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:13.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 >> v1:172.21.15.50:6789/0 conn(0x7feb38158a10 legacy=0x7feb38070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb240039a0 0x7feb24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb24004d00 0x7feb24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38158df0 con 0x7feb24004d00 2023-10-19T10:38:13.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400bb10 con 0x7feb240039a0 2023-10-19T10:38:13.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400bcd0 con 0x7feb380cce20 2023-10-19T10:38:13.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb24004d00 0x7feb24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3d7d0640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb240039a0 0x7feb24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.720+0000 7feb3cfcf640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb3dfd1640 1 -- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 msgr2=0x7feb240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:13.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb380cce20 0x7feb240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb3dfd1640 1 -- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb240039a0 msgr2=0x7feb24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:13.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb240039a0 0x7feb24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb3dfd1640 1 -- 172.21.15.50:0/1643317971 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24003520 con 0x7feb24004d00 2023-10-19T10:38:13.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb24004d00 0x7feb24005180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb300104a0 tx=0x7feb240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8388229daf12ec7d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30038420 con 0x7feb24004d00 2023-10-19T10:38:13.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.721+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30031070 con 0x7feb24004d00 2023-10-19T10:38:13.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb2ffff640 1 -- 172.21.15.50:0/1643317971 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30038780 con 0x7feb24004d00 2023-10-19T10:38:13.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3fa34640 1 -- 172.21.15.50:0/1643317971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb24004d00 msgr2=0x7feb24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3fa34640 1 --2- 172.21.15.50:0/1643317971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb24004d00 0x7feb24005180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb300104a0 tx=0x7feb240057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3dfd1640 1 -- 172.21.15.50:0/1643317971 reap_dead start 2023-10-19T10:38:13.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3fa34640 1 -- 172.21.15.50:0/1643317971 shutdown_connections 2023-10-19T10:38:13.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3fa34640 1 -- 172.21.15.50:0/1643317971 >> 172.21.15.50:0/1643317971 conn(0x7feb380afbe0 msgr2=0x7feb380affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:13.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3fa34640 1 -- 172.21.15.50:0/1643317971 shutdown_connections 2023-10-19T10:38:13.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.722+0000 7feb3fa34640 1 -- 172.21.15.50:0/1643317971 wait complete. 2023-10-19T10:38:13.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.723+0000 7feb3fa34640 1 Processor -- start 2023-10-19T10:38:13.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.723+0000 7feb3fa34640 1 -- start start 2023-10-19T10:38:13.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3fa34640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 0x7feb380b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3fa34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb38153ce0 0x7feb380b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3fa34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb380b0d20 0x7feb380b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3fa34640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380b1220 con 0x7feb38153900 2023-10-19T10:38:13.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3fa34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380b1500 con 0x7feb380b0d20 2023-10-19T10:38:13.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3fa34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb380b17e0 con 0x7feb38153ce0 2023-10-19T10:38:13.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 0x7feb380b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 0x7feb380b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50106/0 (socket says 172.21.15.50:50106) 2023-10-19T10:38:13.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 -- 172.21.15.50:0/2937475805 learned_addr learned my addr 172.21.15.50:0/2937475805 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:13.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3cfcf640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb38153ce0 0x7feb380b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3dfd1640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb380b0d20 0x7feb380b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 -- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb38153ce0 msgr2=0x7feb380b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb38153ce0 0x7feb380b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 -- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb380b0d20 msgr2=0x7feb380b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb380b0d20 0x7feb380b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.724+0000 7feb3d7d0640 1 -- 172.21.15.50:0/2937475805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380b1ac0 con 0x7feb38153900 2023-10-19T10:38:13.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.725+0000 7feb3d7d0640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 0x7feb380b22c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb340011e0 tx=0x7feb34007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.725+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400f9c0 con 0x7feb38153900 2023-10-19T10:38:13.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.725+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3400fb40 con 0x7feb38153900 2023-10-19T10:38:13.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.725+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb380b1da0 con 0x7feb38153900 2023-10-19T10:38:13.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.725+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340133f0 con 0x7feb38153900 2023-10-19T10:38:13.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.725+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb38154110 con 0x7feb38153900 2023-10-19T10:38:13.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.726+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3804e420 con 0x7feb38153900 2023-10-19T10:38:13.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.726+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feb34013550 con 0x7feb38153900 2023-10-19T10:38:13.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.727+0000 7feb2dffb640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feb1c06a050 0x7feb1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:13.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.727+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feb34088b40 con 0x7feb38153900 2023-10-19T10:38:13.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.727+0000 7feb3cfcf640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feb1c06a050 0x7feb1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:13.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.728+0000 7feb3cfcf640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feb1c06a050 0x7feb1c06c510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb381540c0 tx=0x7feb28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:13.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.731+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7feb34053f00 con 0x7feb38153900 2023-10-19T10:38:13.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.907+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb380c1cc0 con 0x7feb38153900 2023-10-19T10:38:13.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.907+0000 7feb2dffb640 1 -- 172.21.15.50:0/2937475805 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb34057520 con 0x7feb38153900 2023-10-19T10:38:13.911 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:13.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feb1c06a050 msgr2=0x7feb1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feb1c06a050 0x7feb1c06c510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb381540c0 tx=0x7feb28009040 comp rx=0 tx=0).stop 2023-10-19T10:38:13.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 msgr2=0x7feb380b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:13.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 0x7feb380b22c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb340011e0 tx=0x7feb34007720 comp rx=0 tx=0).stop 2023-10-19T10:38:13.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 shutdown_connections 2023-10-19T10:38:13.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feb1c06a050 0x7feb1c06c510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb380b0d20 0x7feb380b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb38153ce0 0x7feb380b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 --2- 172.21.15.50:0/2937475805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb38153900 0x7feb380b22c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:13.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 >> 172.21.15.50:0/2937475805 conn(0x7feb380afbe0 msgr2=0x7feb38070090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:13.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 shutdown_connections 2023-10-19T10:38:13.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:13.909+0000 7feb3fa34640 1 -- 172.21.15.50:0/2937475805 wait complete. 2023-10-19T10:38:13.923 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:38:14.032 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.028+0000 7f1038236640 1 Processor -- start 2023-10-19T10:38:14.032 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.029+0000 7f1038236640 1 -- start start 2023-10-19T10:38:14.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.029+0000 7f1038236640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1030158a10 0x7f1030070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.029+0000 7f1038236640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10300cce20 0x7f10300cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.029+0000 7f1038236640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10300cc0f0 con 0x7f1030071130 2023-10-19T10:38:14.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.029+0000 7f1038236640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300cd770 con 0x7f1030158a10 2023-10-19T10:38:14.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.029+0000 7f1038236640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1030158df0 con 0x7f10300cce20 2023-10-19T10:38:14.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1030158a10 0x7f1030070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1030158a10 0x7f1030070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50116/0 (socket says 172.21.15.50:50116) 2023-10-19T10:38:14.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 -- 172.21.15.50:0/77116432 learned_addr learned my addr 172.21.15.50:0/77116432 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:14.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f10367d3640 1 --2- 172.21.15.50:0/77116432 >> v2:172.21.15.183:3300/0 conn(0x7f10300cce20 0x7f10300cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 -- 172.21.15.50:0/77116432 >> v2:172.21.15.183:3300/0 conn(0x7f10300cce20 msgr2=0x7f10300cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:14.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 --2- 172.21.15.50:0/77116432 >> v2:172.21.15.183:3300/0 conn(0x7f10300cce20 0x7f10300cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 -- 172.21.15.50:0/77116432 >> v1:172.21.15.183:6789/0 conn(0x7f1030071130 legacy=0x7f103006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f1035fd2640 1 -- 172.21.15.50:0/77116432 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030070a80 con 0x7f1030158a10 2023-10-19T10:38:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.030+0000 7f10367d3640 1 --2- 172.21.15.50:0/77116432 >> v2:172.21.15.183:3300/0 conn(0x7f10300cce20 0x7f10300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1035fd2640 1 --2- 172.21.15.50:0/77116432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f1030070540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f103006d530 tx=0x7f102802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2def3004ad5728 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1034fd0640 1 -- 172.21.15.50:0/77116432 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028031870 con 0x7f1030158a10 2023-10-19T10:38:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1034fd0640 1 -- 172.21.15.50:0/77116432 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10280319d0 con 0x7f1030158a10 2023-10-19T10:38:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1034fd0640 1 -- 172.21.15.50:0/77116432 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028031d60 con 0x7f1030158a10 2023-10-19T10:38:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 -- 172.21.15.50:0/77116432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 msgr2=0x7f1030070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 --2- 172.21.15.50:0/77116432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f1030070540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f103006d530 tx=0x7f102802b910 comp rx=0 tx=0).stop 2023-10-19T10:38:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 -- 172.21.15.50:0/77116432 shutdown_connections 2023-10-19T10:38:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 --2- 172.21.15.50:0/77116432 >> v2:172.21.15.183:3300/0 conn(0x7f10300cce20 0x7f10300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 --2- 172.21.15.50:0/77116432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f1030070540 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 -- 172.21.15.50:0/77116432 >> 172.21.15.50:0/77116432 conn(0x7f10300afbe0 msgr2=0x7f10300affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 -- 172.21.15.50:0/77116432 shutdown_connections 2023-10-19T10:38:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.031+0000 7f1038236640 1 -- 172.21.15.50:0/77116432 wait complete. 2023-10-19T10:38:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 Processor -- start 2023-10-19T10:38:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 -- start start 2023-10-19T10:38:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1030071130 0x7f10301f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10300cce20 0x7f10300b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f103015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300c1fc0 con 0x7f1030158a10 2023-10-19T10:38:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10301f5ce0 con 0x7f10300cce20 2023-10-19T10:38:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f1038236640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10301f5720 con 0x7f1030071130 2023-10-19T10:38:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f10367d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f103015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f10367d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f103015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50126/0 (socket says 172.21.15.50:50126) 2023-10-19T10:38:14.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.033+0000 7f10367d3640 1 -- 172.21.15.50:0/3537857927 learned_addr learned my addr 172.21.15.50:0/3537857927 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:14.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f1035fd2640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1030071130 0x7f10301f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10357d1640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10300cce20 0x7f10300b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10367d3640 1 -- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1030071130 msgr2=0x7f10301f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10367d3640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1030071130 0x7f10301f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10367d3640 1 -- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10300cce20 msgr2=0x7f10300b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10367d3640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10300cce20 0x7f10300b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10367d3640 1 -- 172.21.15.50:0/3537857927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280094d0 con 0x7f1030158a10 2023-10-19T10:38:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f1035fd2640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1030071130 0x7f10301f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f10367d3640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f103015b000 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1020007cd0 tx=0x7f102000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000f040 con 0x7f1030158a10 2023-10-19T10:38:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102000dda0 con 0x7f1030158a10 2023-10-19T10:38:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10301f5a00 con 0x7f1030158a10 2023-10-19T10:38:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020003bd0 con 0x7f1030158a10 2023-10-19T10:38:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.034+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10301f5360 con 0x7f1030158a10 2023-10-19T10:38:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.035+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10300c2170 con 0x7f1030158a10 2023-10-19T10:38:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.036+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1020003d30 con 0x7f1030158a10 2023-10-19T10:38:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.036+0000 7f102e7fc640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f101c06a050 0x7f101c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.036+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1020088a40 con 0x7f1030158a10 2023-10-19T10:38:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.036+0000 7f1035fd2640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f101c06a050 0x7f101c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.037+0000 7f1035fd2640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f101c06a050 0x7f101c06c510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1028036000 tx=0x7f102803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.041+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f10300c2170 con 0x7f1030158a10 2023-10-19T10:38:14.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.214+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10301f54f0 con 0x7f1030158a10 2023-10-19T10:38:14.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.214+0000 7f102e7fc640 1 -- 172.21.15.50:0/3537857927 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f102008ae60 con 0x7f1030158a10 2023-10-19T10:38:14.218 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:38:14.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f101c06a050 msgr2=0x7f101c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f101c06a050 0x7f101c06c510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1028036000 tx=0x7f102803c040 comp rx=0 tx=0).stop 2023-10-19T10:38:14.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 msgr2=0x7f103015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f103015b000 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1020007cd0 tx=0x7f102000d490 comp rx=0 tx=0).stop 2023-10-19T10:38:14.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 shutdown_connections 2023-10-19T10:38:14.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f101c06a050 0x7f101c06c510 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1030158a10 0x7f103015b000 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10300cce20 0x7f10300b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 --2- 172.21.15.50:0/3537857927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1030071130 0x7f10301f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 >> 172.21.15.50:0/3537857927 conn(0x7f10300afbe0 msgr2=0x7f1030154370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:14.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 shutdown_connections 2023-10-19T10:38:14.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.216+0000 7f1038236640 1 -- 172.21.15.50:0/3537857927 wait complete. 2023-10-19T10:38:14.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-19T10:38:14.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.337+0000 7f24a2beb640 1 Processor -- start 2023-10-19T10:38:14.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a2beb640 1 -- start start 2023-10-19T10:38:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a2beb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0ce020 0x7f249c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a2beb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c15efb0 con 0x7f249c0b21d0 2023-10-19T10:38:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a2beb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c15e3a0 con 0x7f249c0b25b0 2023-10-19T10:38:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a2beb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0ce4b0 con 0x7f249c0ce020 2023-10-19T10:38:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a0987640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f249c0b21d0 0x7f249c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39426/0 (socket says 172.21.15.50:39426) 2023-10-19T10:38:14.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.338+0000 7f24a0987640 1 -- 172.21.15.50:0/3173250011 learned_addr learned my addr 172.21.15.50:0/3173250011 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f24a1188640 1 --2- 172.21.15.50:0/3173250011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0ce020 0x7f249c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816921615 0 0) 0x7f249c15efb0 con 0x7f249c0b21d0 2023-10-19T10:38:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f248c003520 con 0x7f249c0b21d0 2023-10-19T10:38:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564125670 0 0) 0x7f249c15e3a0 con 0x7f249c0b25b0 2023-10-19T10:38:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f249c15efb0 con 0x7f249c0b25b0 2023-10-19T10:38:14.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f24a1188640 1 -- 172.21.15.50:0/3173250011 >> v1:172.21.15.183:6789/0 conn(0x7f249c0b25b0 legacy=0x7f249c06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f24a1188640 1 -- 172.21.15.50:0/3173250011 >> v1:172.21.15.50:6789/0 conn(0x7f249c0b21d0 legacy=0x7f249c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.339+0000 7f24a1188640 1 -- 172.21.15.50:0/3173250011 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c15e680 con 0x7f249c0ce020 2023-10-19T10:38:14.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f24a1188640 1 --2- 172.21.15.50:0/3173250011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0ce020 0x7f249c15b500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2494009dc0 tx=0x7f249402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e191a9dabe20334 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249402ccd0 con 0x7f249c0ce020 2023-10-19T10:38:14.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249402ce30 con 0x7f249c0ce020 2023-10-19T10:38:14.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f249b7fe640 1 -- 172.21.15.50:0/3173250011 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494034640 con 0x7f249c0ce020 2023-10-19T10:38:14.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f24a2beb640 1 -- 172.21.15.50:0/3173250011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0ce020 msgr2=0x7f249c15b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f24a2beb640 1 --2- 172.21.15.50:0/3173250011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0ce020 0x7f249c15b500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2494009dc0 tx=0x7f249402bf20 comp rx=0 tx=0).stop 2023-10-19T10:38:14.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f24a2beb640 1 -- 172.21.15.50:0/3173250011 shutdown_connections 2023-10-19T10:38:14.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f24a2beb640 1 --2- 172.21.15.50:0/3173250011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0ce020 0x7f249c15b500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.340+0000 7f24a2beb640 1 -- 172.21.15.50:0/3173250011 >> 172.21.15.50:0/3173250011 conn(0x7f249c0ccf60 msgr2=0x7f249c0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:14.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.341+0000 7f24a2beb640 1 -- 172.21.15.50:0/3173250011 shutdown_connections 2023-10-19T10:38:14.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.341+0000 7f24a2beb640 1 -- 172.21.15.50:0/3173250011 wait complete. 2023-10-19T10:38:14.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.343+0000 7f24a2beb640 1 Processor -- start 2023-10-19T10:38:14.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.343+0000 7f24a2beb640 1 -- start start 2023-10-19T10:38:14.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.343+0000 7f24a2beb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0b21d0 0x7f249c0b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 0x7f249c1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f249c0ce020 0x7f249c0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c150f30 con 0x7f249c0b25b0 2023-10-19T10:38:14.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0c4310 con 0x7f249c0ce020 2023-10-19T10:38:14.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0c4590 con 0x7f249c0b21d0 2023-10-19T10:38:14.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 0x7f249c1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 0x7f249c1509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50134/0 (socket says 172.21.15.50:50134) 2023-10-19T10:38:14.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 -- 172.21.15.50:0/2682711591 learned_addr learned my addr 172.21.15.50:0/2682711591 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:14.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a1188640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f249c0ce020 0x7f249c0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a0987640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0b21d0 0x7f249c0b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 -- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0b21d0 msgr2=0x7f249c0b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0b21d0 0x7f249c0b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 -- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f249c0ce020 msgr2=0x7f249c0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f249c0ce020 0x7f249c0c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.355 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 -- 172.21.15.50:0/2682711591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494009a70 con 0x7f249c0b25b0 2023-10-19T10:38:14.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f249bfff640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 0x7f249c1509f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f248800c500 tx=0x7f248800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880052d0 con 0x7f249c0b25b0 2023-10-19T10:38:14.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2488005450 con 0x7f249c0b25b0 2023-10-19T10:38:14.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488012870 con 0x7f249c0b25b0 2023-10-19T10:38:14.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c0c4870 con 0x7f249c0b25b0 2023-10-19T10:38:14.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c0c4e30 con 0x7f249c0b25b0 2023-10-19T10:38:14.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.344+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c04e420 con 0x7f249c0b25b0 2023-10-19T10:38:14.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.345+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f24880129d0 con 0x7f249c0b25b0 2023-10-19T10:38:14.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.345+0000 7f24997fa640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f248006a050 0x7f248006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.345+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2488088390 con 0x7f249c0b25b0 2023-10-19T10:38:14.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.346+0000 7f24a0987640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f248006a050 0x7f248006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.346+0000 7f24a0987640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f248006a050 0x7f248006c510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f24900011e0 tx=0x7f2490007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.350+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f24880572a0 con 0x7f249c0b25b0 2023-10-19T10:38:14.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.527+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f249c0c3b60 con 0x7f249c0b25b0 2023-10-19T10:38:14.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.528+0000 7f24997fa640 1 -- 172.21.15.50:0/2682711591 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2488056e30 con 0x7f249c0b25b0 2023-10-19T10:38:14.532 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:38:14.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f248006a050 msgr2=0x7f248006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f248006a050 0x7f248006c510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f24900011e0 tx=0x7f2490007d30 comp rx=0 tx=0).stop 2023-10-19T10:38:14.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 msgr2=0x7f249c1509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 0x7f249c1509f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f248800c500 tx=0x7f248800c9c0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 shutdown_connections 2023-10-19T10:38:14.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f248006a050 0x7f248006c510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f249c0ce020 0x7f249c0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f249c0b25b0 0x7f249c1509f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 --2- 172.21.15.50:0/2682711591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f249c0b21d0 0x7f249c0b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 >> 172.21.15.50:0/2682711591 conn(0x7f249c0ccf60 msgr2=0x7f249c15a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:14.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 shutdown_connections 2023-10-19T10:38:14.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.530+0000 7f24a2beb640 1 -- 172.21.15.50:0/2682711591 wait complete. 2023-10-19T10:38:14.544 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:14.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.646+0000 7fe52fce3640 1 Processor -- start 2023-10-19T10:38:14.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.646+0000 7fe52fce3640 1 -- start start 2023-10-19T10:38:14.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52fce3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5280d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52fce3640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52806aa30 con 0x7fe5280cd920 2023-10-19T10:38:14.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52fce3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52806b2d0 con 0x7fe52815c950 2023-10-19T10:38:14.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52fce3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280ce090 con 0x7fe52815c570 2023-10-19T10:38:14.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52e280640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe5280cd920 0x7fe5280d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39428/0 (socket says 172.21.15.50:39428) 2023-10-19T10:38:14.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52e280640 1 -- 172.21.15.50:0/1992315949 learned_addr learned my addr 172.21.15.50:0/1992315949 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:14.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52da7f640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5280d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991938804 0 0) 0x7fe52806aa30 con 0x7fe5280cd920 2023-10-19T10:38:14.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe510003520 con 0x7fe5280cd920 2023-10-19T10:38:14.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455015892 0 0) 0x7fe52806b2d0 con 0x7fe52815c950 2023-10-19T10:38:14.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe52806aa30 con 0x7fe52815c950 2023-10-19T10:38:14.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423271210 0 0) 0x7fe510003520 con 0x7fe5280cd920 2023-10-19T10:38:14.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 msgr2=0x7fe5280d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:14.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5280d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 >> v1:172.21.15.183:6789/0 conn(0x7fe52815c950 legacy=0x7fe52806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52806ac80 con 0x7fe5280cd920 2023-10-19T10:38:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fe5180019f0 con 0x7fe5280cd920 2023-10-19T10:38:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52da7f640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5280d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 >> v1:172.21.15.50:6789/0 conn(0x7fe5280cd920 legacy=0x7fe5280d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe510003b30 0x7fe510003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe510004d00 0x7fe510005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280ce090 con 0x7fe510004d00 2023-10-19T10:38:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51000ba60 con 0x7fe510003b30 2023-10-19T10:38:14.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51000bc20 con 0x7fe52815c570 2023-10-19T10:38:14.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52e280640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe510004d00 0x7fe510005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52da7f640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe510003b30 0x7fe510003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.648+0000 7fe52d27e640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52e280640 1 -- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 msgr2=0x7fe5100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:14.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52e280640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe52815c570 0x7fe5100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52e280640 1 -- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe510003b30 msgr2=0x7fe510003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:14.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52e280640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe510003b30 0x7fe510003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52e280640 1 -- 172.21.15.50:0/1992315949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200093a0 con 0x7fe510004d00 2023-10-19T10:38:14.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52e280640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe510004d00 0x7fe510005180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe51800a7c0 tx=0x7fe5100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dea0012e3b02c04 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518034630 con 0x7fe510004d00 2023-10-19T10:38:14.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.649+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51802f070 con 0x7fe510004d00 2023-10-19T10:38:14.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52ca7d640 1 -- 172.21.15.50:0/1992315949 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5180349f0 con 0x7fe510004d00 2023-10-19T10:38:14.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52fce3640 1 -- 172.21.15.50:0/1992315949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe510004d00 msgr2=0x7fe510005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52fce3640 1 --2- 172.21.15.50:0/1992315949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe510004d00 0x7fe510005180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe51800a7c0 tx=0x7fe5100057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52e280640 1 -- 172.21.15.50:0/1992315949 reap_dead start 2023-10-19T10:38:14.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52fce3640 1 -- 172.21.15.50:0/1992315949 shutdown_connections 2023-10-19T10:38:14.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52fce3640 1 -- 172.21.15.50:0/1992315949 >> 172.21.15.50:0/1992315949 conn(0x7fe528089ff0 msgr2=0x7fe52809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:14.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52fce3640 1 -- 172.21.15.50:0/1992315949 shutdown_connections 2023-10-19T10:38:14.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.650+0000 7fe52fce3640 1 -- 172.21.15.50:0/1992315949 wait complete. 2023-10-19T10:38:14.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.651+0000 7fe52fce3640 1 Processor -- start 2023-10-19T10:38:14.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.651+0000 7fe52fce3640 1 -- start start 2023-10-19T10:38:14.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52fce3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe528159300 0x7fe5280b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52fce3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5280b2c50 0x7fe5280c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52fce3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 0x7fe52815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52fce3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280c7db0 con 0x7fe5280c6a00 2023-10-19T10:38:14.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52fce3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52815e550 con 0x7fe5280b2c50 2023-10-19T10:38:14.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52fce3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280b30d0 con 0x7fe528159300 2023-10-19T10:38:14.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 0x7fe52815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 0x7fe52815cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50152/0 (socket says 172.21.15.50:50152) 2023-10-19T10:38:14.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 -- 172.21.15.50:0/53292939 learned_addr learned my addr 172.21.15.50:0/53292939 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:14.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52da7f640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe528159300 0x7fe5280b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52d27e640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5280b2c50 0x7fe5280c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 -- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe528159300 msgr2=0x7fe5280b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe528159300 0x7fe5280b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 -- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5280b2c50 msgr2=0x7fe5280c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5280b2c50 0x7fe5280c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.652+0000 7fe52e280640 1 -- 172.21.15.50:0/53292939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280c7230 con 0x7fe5280c6a00 2023-10-19T10:38:14.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.653+0000 7fe52e280640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 0x7fe52815cf70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe51800a7c0 tx=0x7fe518001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.653+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518034630 con 0x7fe5280c6a00 2023-10-19T10:38:14.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.653+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe518034790 con 0x7fe5280c6a00 2023-10-19T10:38:14.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.653+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5280c7510 con 0x7fe5280c6a00 2023-10-19T10:38:14.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.653+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518034b00 con 0x7fe5280c6a00 2023-10-19T10:38:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.653+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5280c7ad0 con 0x7fe5280c6a00 2023-10-19T10:38:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.654+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528073990 con 0x7fe5280c6a00 2023-10-19T10:38:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.654+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe518031030 con 0x7fe5280c6a00 2023-10-19T10:38:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.655+0000 7fe5267fc640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe51406a050 0x7fe51406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.655+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe5180aaa40 con 0x7fe5280c6a00 2023-10-19T10:38:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.655+0000 7fe52da7f640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe51406a050 0x7fe51406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.655+0000 7fe52da7f640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe51406a050 0x7fe51406c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe520009c80 tx=0x7fe520008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.659+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe518079910 con 0x7fe5280c6a00 2023-10-19T10:38:14.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.869+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe528073990 con 0x7fe5280c6a00 2023-10-19T10:38:14.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.869+0000 7fe5267fc640 1 -- 172.21.15.50:0/53292939 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7fe5180a8e60 con 0x7fe5280c6a00 2023-10-19T10:38:14.872 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:14.872 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:38:14.873 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:38:14.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe51406a050 msgr2=0x7fe51406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe51406a050 0x7fe51406c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe520009c80 tx=0x7fe520008040 comp rx=0 tx=0).stop 2023-10-19T10:38:14.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 msgr2=0x7fe52815cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 0x7fe52815cf70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe51800a7c0 tx=0x7fe518001d00 comp rx=0 tx=0).stop 2023-10-19T10:38:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 shutdown_connections 2023-10-19T10:38:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe51406a050 0x7fe51406c510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5280c6a00 0x7fe52815cf70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5280b2c50 0x7fe5280c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 --2- 172.21.15.50:0/53292939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe528159300 0x7fe5280b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 >> 172.21.15.50:0/53292939 conn(0x7fe528089ff0 msgr2=0x7fe5280af320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 shutdown_connections 2023-10-19T10:38:14.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.871+0000 7fe52fce3640 1 -- 172.21.15.50:0/53292939 wait complete. 2023-10-19T10:38:14.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:14.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.988+0000 7f70aa24d640 1 Processor -- start 2023-10-19T10:38:14.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.988+0000 7f70aa24d640 1 -- start start 2023-10-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.988+0000 7f70aa24d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f70a415c570 0x7f70a40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.988+0000 7f70aa24d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c950 0x7f70a40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.988+0000 7f70aa24d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a40cde60 con 0x7f70a406b760 2023-10-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.988+0000 7f70aa24d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a406aa30 con 0x7f70a415c570 2023-10-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70aa24d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a406b2d0 con 0x7f70a415c950 2023-10-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a2ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c950 0x7f70a40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f70a415c570 0x7f70a40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f70a415c570 0x7f70a40d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:58422/0 (socket says 172.21.15.50:58422) 2023-10-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a2ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c950 0x7f70a40cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:48936/0 (socket says 172.21.15.50:48936) 2023-10-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a3fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f70a406b760 0x7f70a40d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:37728/0 (socket says 172.21.15.50:37728) 2023-10-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 -- 172.21.15.50:0/1965047614 learned_addr learned my addr 172.21.15.50:0/1965047614 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 -- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c950 msgr2=0x7f70a40cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 --2- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c950 0x7f70a40cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 -- 172.21.15.50:0/1965047614 >> v1:172.21.15.183:6789/0 conn(0x7f70a406b760 legacy=0x7f70a40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.989+0000 7f70a37fe640 1 -- 172.21.15.50:0/1965047614 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40ce090 con 0x7f70a415c570 2023-10-19T10:38:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.990+0000 7f70a37fe640 1 --2- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c570 0x7f70a40d0e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7094009850 tx=0x7f709402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a539e90264fc173f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.990+0000 7f70a27fc640 1 -- 172.21.15.50:0/1965047614 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094004230 con 0x7f70a415c570 2023-10-19T10:38:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.990+0000 7f70a27fc640 1 -- 172.21.15.50:0/1965047614 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094004390 con 0x7f70a415c570 2023-10-19T10:38:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.990+0000 7f70a27fc640 1 -- 172.21.15.50:0/1965047614 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094031830 con 0x7f70a415c570 2023-10-19T10:38:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 -- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c570 msgr2=0x7f70a40d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 --2- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c570 0x7f70a40d0e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7094009850 tx=0x7f709402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 -- 172.21.15.50:0/1965047614 shutdown_connections 2023-10-19T10:38:15.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 --2- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c950 0x7f70a40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 --2- 172.21.15.50:0/1965047614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c570 0x7f70a40d0e30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 -- 172.21.15.50:0/1965047614 >> 172.21.15.50:0/1965047614 conn(0x7f70a4089ff0 msgr2=0x7f70a409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 -- 172.21.15.50:0/1965047614 shutdown_connections 2023-10-19T10:38:15.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.991+0000 7f70aa24d640 1 -- 172.21.15.50:0/1965047614 wait complete. 2023-10-19T10:38:15.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.992+0000 7f70aa24d640 1 Processor -- start 2023-10-19T10:38:15.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.992+0000 7f70aa24d640 1 -- start start 2023-10-19T10:38:15.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70aa24d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 0x7f70a415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70aa24d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c570 0x7f70a415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70aa24d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c950 0x7f70a415f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70aa24d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40c7f30 con 0x7f70a406b760 2023-10-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70aa24d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40c7600 con 0x7f70a415c950 2023-10-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70aa24d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40c78e0 con 0x7f70a415c570 2023-10-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 0x7f70a415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 0x7f70a415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50162/0 (socket says 172.21.15.50:50162) 2023-10-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 -- 172.21.15.50:0/4281740176 learned_addr learned my addr 172.21.15.50:0/4281740176 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a3fff640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c950 0x7f70a415f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a2ffd640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c570 0x7f70a415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 -- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c570 msgr2=0x7f70a415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c570 0x7f70a415a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 -- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c950 msgr2=0x7f70a415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c950 0x7f70a415f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.993+0000 7f70a37fe640 1 -- 172.21.15.50:0/4281740176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009460 con 0x7f70a406b760 2023-10-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.994+0000 7f70a37fe640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 0x7f70a415a410 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f709402b910 tx=0x7f70940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.994+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094031630 con 0x7f70a406b760 2023-10-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.994+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709403f070 con 0x7f70a406b760 2023-10-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.994+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a40c7bc0 con 0x7f70a406b760 2023-10-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.994+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70940319b0 con 0x7f70a406b760 2023-10-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.994+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a40c83c0 con 0x7f70a406b760 2023-10-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.995+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a40c7d50 con 0x7f70a406b760 2023-10-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.995+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7094041030 con 0x7f70a406b760 2023-10-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.996+0000 7f708bfff640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f709006a050 0x7f709006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.996+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70940aa9d0 con 0x7f70a406b760 2023-10-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.996+0000 7f70a2ffd640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f709006a050 0x7f709006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:14.996+0000 7f70a2ffd640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f709006a050 0x7f709006c510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70a40c9520 tx=0x7f7098009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.000+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7094075f60 con 0x7f70a406b760 2023-10-19T10:38:15.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.179+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70a4073990 con 0x7f70a406b760 2023-10-19T10:38:15.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.180+0000 7f708bfff640 1 -- 172.21.15.50:0/4281740176 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f7094079580 con 0x7f70a406b760 2023-10-19T10:38:15.183 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:15.183 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:15.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f709006a050 msgr2=0x7f709006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f709006a050 0x7f709006c510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70a40c9520 tx=0x7f7098009190 comp rx=0 tx=0).stop 2023-10-19T10:38:15.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 msgr2=0x7f70a415a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 0x7f70a415a410 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f709402b910 tx=0x7f70940046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 shutdown_connections 2023-10-19T10:38:15.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f709006a050 0x7f709006c510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a415c950 0x7f70a415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f70a415c570 0x7f70a415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 --2- 172.21.15.50:0/4281740176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70a406b760 0x7f70a415a410 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 >> 172.21.15.50:0/4281740176 conn(0x7f70a4089ff0 msgr2=0x7f70a415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 shutdown_connections 2023-10-19T10:38:15.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.182+0000 7f70aa24d640 1 -- 172.21.15.50:0/4281740176 wait complete. 2023-10-19T10:38:15.196 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-19T10:38:15.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:15.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.300+0000 7f3257985640 1 Processor -- start 2023-10-19T10:38:15.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.300+0000 7f3257985640 1 -- start start 2023-10-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.301+0000 7f3257985640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32501607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.301+0000 7f3257985640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325015d6e0 con 0x7f325015ddf0 2023-10-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.301+0000 7f3257985640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325015f030 con 0x7f32500b1f30 2023-10-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.301+0000 7f3257985640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325015cdb0 con 0x7f3250160340 2023-10-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.301+0000 7f3255f22640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f325015ddf0 0x7f325015e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39440/0 (socket says 172.21.15.50:39440) 2023-10-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.301+0000 7f3255f22640 1 -- 172.21.15.50:0/2042516522 learned_addr learned my addr 172.21.15.50:0/2042516522 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576490386 0 0) 0x7f325015d6e0 con 0x7f325015ddf0 2023-10-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3238003520 con 0x7f325015ddf0 2023-10-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3254f20640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32501607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442950716 0 0) 0x7f3238003520 con 0x7f325015ddf0 2023-10-19T10:38:15.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 msgr2=0x7f32501607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:15.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32501607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 >> v1:172.21.15.183:6789/0 conn(0x7f32500b1f30 legacy=0x7f32500b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325015d090 con 0x7f325015ddf0 2023-10-19T10:38:15.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f32400019f0 con 0x7f325015ddf0 2023-10-19T10:38:15.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 >> v1:172.21.15.50:6789/0 conn(0x7f325015ddf0 legacy=0x7f325015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32380039a0 0x7f3238003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3238004d00 0x7f3238005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325015cdb0 con 0x7f32380039a0 2023-10-19T10:38:15.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323800bab0 con 0x7f3238004d00 2023-10-19T10:38:15.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323800bc70 con 0x7f3250160340 2023-10-19T10:38:15.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255721640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32380039a0 0x7f3238003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3254f20640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255f22640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3238004d00 0x7f3238005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255721640 1 -- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 msgr2=0x7f32380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:15.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255721640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255721640 1 -- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3238004d00 msgr2=0x7f3238005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:15.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255721640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3238004d00 0x7f3238005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.302+0000 7f3255721640 1 -- 172.21.15.50:0/2042516522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238003520 con 0x7f32380039a0 2023-10-19T10:38:15.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.303+0000 7f3254f20640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3250160340 0x7f32380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.303+0000 7f3255721640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32380039a0 0x7f3238003dc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3248001750 tx=0x7f32380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f504c0f8437439 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.303+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32480052d0 con 0x7f32380039a0 2023-10-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.303+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324802e050 con 0x7f32380039a0 2023-10-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.303+0000 7f324ffff640 1 -- 172.21.15.50:0/2042516522 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32480035f0 con 0x7f32380039a0 2023-10-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3257985640 1 -- 172.21.15.50:0/2042516522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32380039a0 msgr2=0x7f3238003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3257985640 1 --2- 172.21.15.50:0/2042516522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32380039a0 0x7f3238003dc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3248001750 tx=0x7f32380057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3255f22640 1 -- 172.21.15.50:0/2042516522 reap_dead start 2023-10-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3257985640 1 -- 172.21.15.50:0/2042516522 shutdown_connections 2023-10-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3257985640 1 -- 172.21.15.50:0/2042516522 >> 172.21.15.50:0/2042516522 conn(0x7f32500ccf30 msgr2=0x7f32500cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3257985640 1 -- 172.21.15.50:0/2042516522 shutdown_connections 2023-10-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.304+0000 7f3257985640 1 -- 172.21.15.50:0/2042516522 wait complete. 2023-10-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.305+0000 7f3257985640 1 Processor -- start 2023-10-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.305+0000 7f3257985640 1 -- start start 2023-10-19T10:38:15.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3257985640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32380039a0 0x7f32501f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3257985640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 0x7f32500c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3257985640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32501f49e0 0x7f325006b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3257985640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325006b220 con 0x7f32500c7740 2023-10-19T10:38:15.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3257985640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325006a910 con 0x7f32380039a0 2023-10-19T10:38:15.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3257985640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32501f3360 con 0x7f32501f49e0 2023-10-19T10:38:15.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 0x7f32500c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 0x7f32500c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50186/0 (socket says 172.21.15.50:50186) 2023-10-19T10:38:15.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 -- 172.21.15.50:0/140190954 learned_addr learned my addr 172.21.15.50:0/140190954 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:15.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3255721640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32380039a0 0x7f32501f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3255f22640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32501f49e0 0x7f325006b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 -- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32501f49e0 msgr2=0x7f325006b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32501f49e0 0x7f325006b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 -- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32380039a0 msgr2=0x7f32501f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32380039a0 0x7f32501f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.306+0000 7f3254f20640 1 -- 172.21.15.50:0/140190954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500ca030 con 0x7f32500c7740 2023-10-19T10:38:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.307+0000 7f3254f20640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 0x7f32500c9af0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3244011700 tx=0x7f3244011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.307+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400f9c0 con 0x7f32500c7740 2023-10-19T10:38:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.307+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324400fb20 con 0x7f32500c7740 2023-10-19T10:38:15.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.307+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32500ca2f0 con 0x7f32500c7740 2023-10-19T10:38:15.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.307+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400fed0 con 0x7f32500c7740 2023-10-19T10:38:15.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.307+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325006ae40 con 0x7f32500c7740 2023-10-19T10:38:15.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.308+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3250073990 con 0x7f32500c7740 2023-10-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.308+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3244015460 con 0x7f32500c7740 2023-10-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.309+0000 7f324dffb640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f323c06a080 0x7f323c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.309+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3244089010 con 0x7f32500c7740 2023-10-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.309+0000 7f3255721640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f323c06a080 0x7f323c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.310+0000 7f3255721640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f323c06a080 0x7f323c06c540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3248033000 tx=0x7f3248009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.313+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3244054460 con 0x7f32500c7740 2023-10-19T10:38:15.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.525+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32500b2e50 con 0x7f32500c7740 2023-10-19T10:38:15.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.525+0000 7f324dffb640 1 -- 172.21.15.50:0/140190954 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7f3244057a80 con 0x7f32500c7740 2023-10-19T10:38:15.529 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:15.530 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:38:15.532 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:38:15.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.527+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f323c06a080 msgr2=0x7f323c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.527+0000 7f3257985640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f323c06a080 0x7f323c06c540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3248033000 tx=0x7f3248009030 comp rx=0 tx=0).stop 2023-10-19T10:38:15.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.527+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 msgr2=0x7f32500c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.527+0000 7f3257985640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 0x7f32500c9af0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3244011700 tx=0x7f3244011bc0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 shutdown_connections 2023-10-19T10:38:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f323c06a080 0x7f323c06c540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32501f49e0 0x7f325006b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32500c7740 0x7f32500c9af0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 --2- 172.21.15.50:0/140190954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32380039a0 0x7f32501f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 >> 172.21.15.50:0/140190954 conn(0x7f32500ccf30 msgr2=0x7f32500d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 shutdown_connections 2023-10-19T10:38:15.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.528+0000 7f3257985640 1 -- 172.21.15.50:0/140190954 wait complete. 2023-10-19T10:38:15.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.646+0000 7fb0dd51f640 1 Processor -- start 2023-10-19T10:38:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.646+0000 7fb0dd51f640 1 -- start start 2023-10-19T10:38:15.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0dd51f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb0d80d2db0 0x7fb0d815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0dd51f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d806aa30 con 0x7fb0d815c570 2023-10-19T10:38:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0dd51f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d806b2d0 con 0x7fb0d80d0e30 2023-10-19T10:38:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0dd51f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d815ce90 con 0x7fb0d80d2db0 2023-10-19T10:38:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb0d80d2db0 0x7fb0d815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d6ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb0d815c570 0x7fb0d806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39452/0 (socket says 172.21.15.50:39452) 2023-10-19T10:38:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d6ffd640 1 -- 172.21.15.50:0/3358621412 learned_addr learned my addr 172.21.15.50:0/3358621412 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d77fe640 1 -- 172.21.15.50:0/3358621412 >> v1:172.21.15.183:6789/0 conn(0x7fb0d80d0e30 legacy=0x7fb0d80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d77fe640 1 -- 172.21.15.50:0/3358621412 >> v1:172.21.15.50:6789/0 conn(0x7fb0d815c570 legacy=0x7fb0d806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d5ffb640 1 -- 172.21.15.50:0/3358621412 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138052940 0 0) 0x7fb0d806aa30 con 0x7fb0d815c570 2023-10-19T10:38:15.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.647+0000 7fb0d77fe640 1 -- 172.21.15.50:0/3358621412 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d80d1270 con 0x7fb0d80d2db0 2023-10-19T10:38:15.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.648+0000 7fb0d77fe640 1 --2- 172.21.15.50:0/3358621412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815c950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009d90 tx=0x7fb0d002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc510516650bde3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.648+0000 7fb0d5ffb640 1 -- 172.21.15.50:0/3358621412 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d002cc60 con 0x7fb0d80d2db0 2023-10-19T10:38:15.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.648+0000 7fb0d5ffb640 1 -- 172.21.15.50:0/3358621412 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d002cdc0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.648+0000 7fb0d5ffb640 1 -- 172.21.15.50:0/3358621412 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0034660 con 0x7fb0d80d2db0 2023-10-19T10:38:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 -- 172.21.15.50:0/3358621412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 msgr2=0x7fb0d815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/3358621412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815c950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009d90 tx=0x7fb0d002beb0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 -- 172.21.15.50:0/3358621412 shutdown_connections 2023-10-19T10:38:15.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/3358621412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815c950 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 -- 172.21.15.50:0/3358621412 >> 172.21.15.50:0/3358621412 conn(0x7fb0d8089ff0 msgr2=0x7fb0d809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 -- 172.21.15.50:0/3358621412 shutdown_connections 2023-10-19T10:38:15.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.649+0000 7fb0dd51f640 1 -- 172.21.15.50:0/3358621412 wait complete. 2023-10-19T10:38:15.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.650+0000 7fb0dd51f640 1 Processor -- start 2023-10-19T10:38:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.650+0000 7fb0dd51f640 1 -- start start 2023-10-19T10:38:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0dd51f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0d80d0e30 0x7fb0d81f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0dd51f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0dd51f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0d815c570 0x7fb0d80cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0dd51f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d815e640 con 0x7fb0d80d2db0 2023-10-19T10:38:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0dd51f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d80cf2f0 con 0x7fb0d815c570 2023-10-19T10:38:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0dd51f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d81f5db0 con 0x7fb0d80d0e30 2023-10-19T10:38:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50198/0 (socket says 172.21.15.50:50198) 2023-10-19T10:38:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 -- 172.21.15.50:0/1790440645 learned_addr learned my addr 172.21.15.50:0/1790440645 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d77fe640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0d815c570 0x7fb0d80cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d6ffd640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0d80d0e30 0x7fb0d81f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 -- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0d80d0e30 msgr2=0x7fb0d81f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0d80d0e30 0x7fb0d81f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 -- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0d815c570 msgr2=0x7fb0d80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0d815c570 0x7fb0d80cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 -- 172.21.15.50:0/1790440645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0009a70 con 0x7fb0d80d2db0 2023-10-19T10:38:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d77fe640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0d815c570 0x7fb0d80cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.651+0000 7fb0d67fc640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815b6b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c400 tx=0x7fb0cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.652+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0052d0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.652+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc005430 con 0x7fb0d80d2db0 2023-10-19T10:38:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.652+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d81f6090 con 0x7fb0d80d2db0 2023-10-19T10:38:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.652+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc003830 con 0x7fb0d80d2db0 2023-10-19T10:38:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.652+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d814c530 con 0x7fb0d80d2db0 2023-10-19T10:38:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.653+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d815e4a0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.653+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0cc003990 con 0x7fb0d80d2db0 2023-10-19T10:38:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.653+0000 7fb0bf7fe640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb0c406a050 0x7fb0c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.654+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb0cc087ef0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.654+0000 7fb0d6ffd640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb0c406a050 0x7fb0c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.654+0000 7fb0d6ffd640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb0c406a050 0x7fb0c406c510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80010e0 tx=0x7fb0c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.658+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb0d815e4a0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.868+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0d815e4a0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.868+0000 7fb0bf7fe640 1 -- 172.21.15.50:0/1790440645 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4877 (secure 0 0 0) 0x7fb0d815e4a0 con 0x7fb0d80d2db0 2023-10-19T10:38:15.872 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:15.872 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2847312710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":2847312710},{"type":"v1","addr":"172.21.15.50:6835","nonce":2847312710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1011955576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1011955576},{"type":"v1","addr":"172.21.15.183:6836","nonce":1011955576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2499585140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2499585140},{"type":"v1","addr":"172.21.15.50:6834","nonce":2499585140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:36:54.662444+0000","modified":"2023-10-19T10:37:18.122269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6837/4204689263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":4204689263},{"type":"v1","addr":"172.21.15.183:6837","nonce":4204689263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-19T10:38:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2023-10-19T10:38:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb0c406a050 msgr2=0x7fb0c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb0c406a050 0x7fb0c406c510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80010e0 tx=0x7fb0c8009040 comp rx=0 tx=0).stop 2023-10-19T10:38:15.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 msgr2=0x7fb0d815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815b6b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c400 tx=0x7fb0cc00c8c0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 shutdown_connections 2023-10-19T10:38:15.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb0c406a050 0x7fb0c406c510 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0d815c570 0x7fb0d80cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d80d2db0 0x7fb0d815b6b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 --2- 172.21.15.50:0/1790440645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0d80d0e30 0x7fb0d81f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 >> 172.21.15.50:0/1790440645 conn(0x7fb0d8089ff0 msgr2=0x7fb0d80d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 shutdown_connections 2023-10-19T10:38:15.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.870+0000 7fb0dd51f640 1 -- 172.21.15.50:0/1790440645 wait complete. 2023-10-19T10:38:15.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:15.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.987+0000 7f33b8458640 1 Processor -- start 2023-10-19T10:38:15.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.987+0000 7f33b8458640 1 -- start start 2023-10-19T10:38:15.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b8458640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f33b00d2db0 0x7f33b015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:15.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b8458640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b006aa30 con 0x7f33b00d0e30 2023-10-19T10:38:15.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b8458640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b006b2d0 con 0x7f33b015c570 2023-10-19T10:38:15.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b8458640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b015ce90 con 0x7f33b00d2db0 2023-10-19T10:38:15.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b69f5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f33b00d2db0 0x7f33b015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:15.992 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b59f3640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f33b00d0e30 0x7f33b00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39464/0 (socket says 172.21.15.50:39464) 2023-10-19T10:38:15.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.988+0000 7f33b59f3640 1 -- 172.21.15.50:0/3817438306 learned_addr learned my addr 172.21.15.50:0/3817438306 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:15.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.989+0000 7f33b69f5640 1 -- 172.21.15.50:0/3817438306 >> v1:172.21.15.183:6789/0 conn(0x7f33b015c570 legacy=0x7f33b006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.989+0000 7f33b69f5640 1 -- 172.21.15.50:0/3817438306 >> v1:172.21.15.50:6789/0 conn(0x7f33b00d0e30 legacy=0x7f33b00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.989+0000 7f33b69f5640 1 -- 172.21.15.50:0/3817438306 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b00d1270 con 0x7f33b00d2db0 2023-10-19T10:38:15.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.989+0000 7f33b51f2640 1 -- 172.21.15.50:0/3817438306 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605209185 0 0) 0x7f33b006aa30 con 0x7f33b00d0e30 2023-10-19T10:38:15.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.989+0000 7f33b69f5640 1 --2- 172.21.15.50:0/3817438306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b015c950 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097a0 tx=0x7f33a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccdddd1cc050450 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:15.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b51f2640 1 -- 172.21.15.50:0/3817438306 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8031830 con 0x7f33b00d2db0 2023-10-19T10:38:15.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b51f2640 1 -- 172.21.15.50:0/3817438306 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a8031990 con 0x7f33b00d2db0 2023-10-19T10:38:15.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b51f2640 1 -- 172.21.15.50:0/3817438306 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8031d20 con 0x7f33b00d2db0 2023-10-19T10:38:15.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 -- 172.21.15.50:0/3817438306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 msgr2=0x7f33b015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:15.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 --2- 172.21.15.50:0/3817438306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b015c950 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097a0 tx=0x7f33a802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 -- 172.21.15.50:0/3817438306 shutdown_connections 2023-10-19T10:38:15.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 --2- 172.21.15.50:0/3817438306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b015c950 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:15.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 -- 172.21.15.50:0/3817438306 >> 172.21.15.50:0/3817438306 conn(0x7f33b0089ff0 msgr2=0x7f33b009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:15.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 -- 172.21.15.50:0/3817438306 shutdown_connections 2023-10-19T10:38:16.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.990+0000 7f33b8458640 1 -- 172.21.15.50:0/3817438306 wait complete. 2023-10-19T10:38:16.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.991+0000 7f33b8458640 1 Processor -- start 2023-10-19T10:38:16.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.991+0000 7f33b8458640 1 -- start start 2023-10-19T10:38:16.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b8458640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33b00d0e30 0x7f33b00c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b8458640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b00b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b8458640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33b015c570 0x7f33b00b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b8458640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00c6ef0 con 0x7f33b00d2db0 2023-10-19T10:38:16.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b8458640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b015e310 con 0x7f33b015c570 2023-10-19T10:38:16.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b8458640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00c62e0 con 0x7f33b00d0e30 2023-10-19T10:38:16.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b00b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b00b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50224/0 (socket says 172.21.15.50:50224) 2023-10-19T10:38:16.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 -- 172.21.15.50:0/257589102 learned_addr learned my addr 172.21.15.50:0/257589102 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:16.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 -- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33b00d0e30 msgr2=0x7f33b00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33b00d0e30 0x7f33b00c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 -- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33b015c570 msgr2=0x7f33b00b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.992+0000 7f33b59f3640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33b015c570 0x7f33b00b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33b59f3640 1 -- 172.21.15.50:0/257589102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009400 con 0x7f33b00d2db0 2023-10-19T10:38:16.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33b59f3640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b00b66c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c420 tx=0x7f33a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00074e0 con 0x7f33b00d2db0 2023-10-19T10:38:16.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a0007640 con 0x7f33b00d2db0 2023-10-19T10:38:16.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b00c65c0 con 0x7f33b00d2db0 2023-10-19T10:38:16.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004a00 con 0x7f33b00d2db0 2023-10-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.993+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b00c6b80 con 0x7f33b00d2db0 2023-10-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.994+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b0073990 con 0x7f33b00d2db0 2023-10-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.994+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33a0004b60 con 0x7f33b00d2db0 2023-10-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.995+0000 7f33a6ffd640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f339406a050 0x7f339406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.995+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f33a00886f0 con 0x7f33b00d2db0 2023-10-19T10:38:16.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.995+0000 7f33b61f4640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f339406a050 0x7f339406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.996+0000 7f33b61f4640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f339406a050 0x7f339406c510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33ac001330 tx=0x7f33ac007d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:15.999+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f33a0053ab0 con 0x7f33b00d2db0 2023-10-19T10:38:16.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.173+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33b00c9270 con 0x7f33b00d2db0 2023-10-19T10:38:16.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.174+0000 7f33a6ffd640 1 -- 172.21.15.50:0/257589102 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20178 (secure 0 0 0) 0x7f33a00570d0 con 0x7f33b00d2db0 2023-10-19T10:38:16.177 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:16.178 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":29,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:37:36.203921+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:36:53.037250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-19T10:36:53.634564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-19T10:36:57.357722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:16.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f339406a050 msgr2=0x7f339406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f339406a050 0x7f339406c510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33ac001330 tx=0x7f33ac007d20 comp rx=0 tx=0).stop 2023-10-19T10:38:16.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 msgr2=0x7f33b00b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b00b66c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c420 tx=0x7f33a000c8e0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 shutdown_connections 2023-10-19T10:38:16.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f339406a050 0x7f339406c510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33b015c570 0x7f33b00b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33b00d2db0 0x7f33b00b66c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 --2- 172.21.15.50:0/257589102 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33b00d0e30 0x7f33b00c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 >> 172.21.15.50:0/257589102 conn(0x7f33b0089ff0 msgr2=0x7f33b00d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:16.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 shutdown_connections 2023-10-19T10:38:16.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.176+0000 7f33b8458640 1 -- 172.21.15.50:0/257589102 wait complete. 2023-10-19T10:38:16.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-19T10:38:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.295+0000 7f8acddd9640 1 Processor -- start 2023-10-19T10:38:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.295+0000 7f8acddd9640 1 -- start start 2023-10-19T10:38:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8acddd9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ac80cc910 0x7f8ac80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8acddd9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80d0000 con 0x7f8ac815b3b0 2023-10-19T10:38:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8acddd9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac8070a20 con 0x7f8ac8070eb0 2023-10-19T10:38:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8acddd9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac815b840 con 0x7f8ac80cc910 2023-10-19T10:38:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ac80cc910 0x7f8ac80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac6ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f8ac815b3b0 0x7f8ac80cb4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39466/0 (socket says 172.21.15.50:39466) 2023-10-19T10:38:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac6ffd640 1 -- 172.21.15.50:0/1372091517 learned_addr learned my addr 172.21.15.50:0/1372091517 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ac80cc910 0x7f8ac80cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50234/0 (socket says 172.21.15.50:50234) 2023-10-19T10:38:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac77fe640 1 -- 172.21.15.50:0/1372091517 >> v1:172.21.15.183:6789/0 conn(0x7f8ac8070eb0 legacy=0x7f8ac80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac77fe640 1 -- 172.21.15.50:0/1372091517 >> v1:172.21.15.50:6789/0 conn(0x7f8ac815b3b0 legacy=0x7f8ac80cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac77fe640 1 -- 172.21.15.50:0/1372091517 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac80cf760 con 0x7f8ac80cc910 2023-10-19T10:38:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.296+0000 7f8ac67fc640 1 -- 172.21.15.50:0/1372091517 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704221384 0 0) 0x7f8ab8001110 con 0x7f8ac815b3b0 2023-10-19T10:38:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.297+0000 7f8ac77fe640 1 --2- 172.21.15.50:0/1372091517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac80cc910 0x7f8ac80cf060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0097a0 tx=0x7f8abc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdc1cbef858cdca server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.297+0000 7f8ac67fc640 1 -- 172.21.15.50:0/1372091517 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc030610 con 0x7f8ac80cc910 2023-10-19T10:38:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.297+0000 7f8ac67fc640 1 -- 172.21.15.50:0/1372091517 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc030770 con 0x7f8ac80cc910 2023-10-19T10:38:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.297+0000 7f8ac67fc640 1 -- 172.21.15.50:0/1372091517 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc030b00 con 0x7f8ac80cc910 2023-10-19T10:38:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 -- 172.21.15.50:0/1372091517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac80cc910 msgr2=0x7f8ac80cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 --2- 172.21.15.50:0/1372091517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac80cc910 0x7f8ac80cf060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0097a0 tx=0x7f8abc02b960 comp rx=0 tx=0).stop 2023-10-19T10:38:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 -- 172.21.15.50:0/1372091517 shutdown_connections 2023-10-19T10:38:16.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 --2- 172.21.15.50:0/1372091517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac80cc910 0x7f8ac80cf060 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 -- 172.21.15.50:0/1372091517 >> 172.21.15.50:0/1372091517 conn(0x7f8ac806c9a0 msgr2=0x7f8ac806cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:16.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 -- 172.21.15.50:0/1372091517 shutdown_connections 2023-10-19T10:38:16.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.298+0000 7f8acddd9640 1 -- 172.21.15.50:0/1372091517 wait complete. 2023-10-19T10:38:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.299+0000 7f8acddd9640 1 Processor -- start 2023-10-19T10:38:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.299+0000 7f8acddd9640 1 -- start start 2023-10-19T10:38:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8acddd9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 0x7f8ac814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8acddd9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ac80cc910 0x7f8ac80b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8acddd9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ac815b3b0 0x7f8ac80b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8acddd9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80b1d30 con 0x7f8ac8070eb0 2023-10-19T10:38:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8acddd9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80b2010 con 0x7f8ac80cc910 2023-10-19T10:38:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8acddd9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80b22f0 con 0x7f8ac815b3b0 2023-10-19T10:38:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 0x7f8ac814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 0x7f8ac814c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50240/0 (socket says 172.21.15.50:50240) 2023-10-19T10:38:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 -- 172.21.15.50:0/965888929 learned_addr learned my addr 172.21.15.50:0/965888929 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac6ffd640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ac80cc910 0x7f8ac80b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac7fff640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ac815b3b0 0x7f8ac80b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 -- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ac815b3b0 msgr2=0x7f8ac80b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ac815b3b0 0x7f8ac80b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 -- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ac80cc910 msgr2=0x7f8ac80b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ac80cc910 0x7f8ac80b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 -- 172.21.15.50:0/965888929 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc009400 con 0x7f8ac8070eb0 2023-10-19T10:38:16.312 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac6ffd640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ac80cc910 0x7f8ac80b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8ac77fe640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 0x7f8ac814c690 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0098c0 tx=0x7f8abc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.300+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc03e040 con 0x7f8ac8070eb0 2023-10-19T10:38:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.301+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc039cb0 con 0x7f8ac8070eb0 2023-10-19T10:38:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.301+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac80b25d0 con 0x7f8ac8070eb0 2023-10-19T10:38:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.301+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc044630 con 0x7f8ac8070eb0 2023-10-19T10:38:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.301+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac81f0af0 con 0x7f8ac8070eb0 2023-10-19T10:38:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.302+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac80b2760 con 0x7f8ac8070eb0 2023-10-19T10:38:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.302+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8abc04d3b0 con 0x7f8ac8070eb0 2023-10-19T10:38:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.302+0000 7f8aaffff640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8ab406a080 0x7f8ab406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.302+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8abc0aaf30 con 0x7f8ac8070eb0 2023-10-19T10:38:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.303+0000 7f8ac6ffd640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8ab406a080 0x7f8ab406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.303+0000 7f8ac6ffd640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8ab406a080 0x7f8ab406c540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8007900 tx=0x7f8ab8007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.307+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f8abc0762f0 con 0x7f8ac8070eb0 2023-10-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.519+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8ac8073990 con 0x7f8ac8070eb0 2023-10-19T10:38:16.522 INFO:tasks.ceph.mon.a.smithi050.stderr:2023-10-19T10:38:16.520+0000 7f8abc1be640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.525+0000 7f8aaffff640 1 -- 172.21.15.50:0/965888929 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8abc079910 con 0x7f8ac8070eb0 2023-10-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8ab406a080 msgr2=0x7f8ab406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8ab406a080 0x7f8ab406c540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8007900 tx=0x7f8ab8007ad0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 msgr2=0x7f8ac814c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 0x7f8ac814c690 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0098c0 tx=0x7f8abc004730 comp rx=0 tx=0).stop 2023-10-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 shutdown_connections 2023-10-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8ab406a080 0x7f8ab406c540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ac815b3b0 0x7f8ac80b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ac80cc910 0x7f8ac80b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 --2- 172.21.15.50:0/965888929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac8070eb0 0x7f8ac814c690 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 >> 172.21.15.50:0/965888929 conn(0x7f8ac806c9a0 msgr2=0x7f8ac8069800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 shutdown_connections 2023-10-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.528+0000 7f8acddd9640 1 -- 172.21.15.50:0/965888929 wait complete. 2023-10-19T10:38:16.533 INFO:tasks.ceph.mds.b.smithi183.stderr: -5953> 2023-10-19T10:36:50.270+0000 7f7bf0d24640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-19T10:38:16.534 INFO:tasks.ceph.mds.b.smithi183.stderr: -4717> 2023-10-19T10:37:01.284+0000 7f7bead18640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-19T10:38:16.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-19T10:38:16.563 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-10-19T10:38:16.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.647+0000 7fcf7e12d640 1 Processor -- start 2023-10-19T10:38:16.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.647+0000 7fcf7e12d640 1 -- start start 2023-10-19T10:38:16.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf7e12d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7815c620 0x7fcf780b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf7e12d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcf780cfb00 0x7fcf780d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf7e12d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf780cda30 con 0x7fcf780b82f0 2023-10-19T10:38:16.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf7e12d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf780ccc10 con 0x7fcf780cfb00 2023-10-19T10:38:16.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf7e12d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78161110 con 0x7fcf7815c620 2023-10-19T10:38:16.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcf780cfb00 0x7fcf780d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcf780cfb00 0x7fcf780d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50254/0 (socket says 172.21.15.50:50254) 2023-10-19T10:38:16.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 -- 172.21.15.50:0/2575001342 learned_addr learned my addr 172.21.15.50:0/2575001342 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:16.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf777fe640 1 --2- 172.21.15.50:0/2575001342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7815c620 0x7fcf780b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 -- 172.21.15.50:0/2575001342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7815c620 msgr2=0x7fcf780b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:16.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 --2- 172.21.15.50:0/2575001342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7815c620 0x7fcf780b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 -- 172.21.15.50:0/2575001342 >> v1:172.21.15.183:6789/0 conn(0x7fcf780b82f0 legacy=0x7fcf780cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.648+0000 7fcf77fff640 1 -- 172.21.15.50:0/2575001342 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780cce20 con 0x7fcf780cfb00 2023-10-19T10:38:16.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.649+0000 7fcf77fff640 1 --2- 172.21.15.50:0/2575001342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf780cfb00 0x7fcf780d0b00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009e50 tx=0x7fcf7002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868c0e740fe3df16 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.649+0000 7fcf767fc640 1 -- 172.21.15.50:0/2575001342 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7002cb00 con 0x7fcf780cfb00 2023-10-19T10:38:16.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.649+0000 7fcf767fc640 1 -- 172.21.15.50:0/2575001342 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7002cc60 con 0x7fcf780cfb00 2023-10-19T10:38:16.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.649+0000 7fcf767fc640 1 -- 172.21.15.50:0/2575001342 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700343f0 con 0x7fcf780cfb00 2023-10-19T10:38:16.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 -- 172.21.15.50:0/2575001342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf780cfb00 msgr2=0x7fcf780d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/2575001342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf780cfb00 0x7fcf780d0b00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009e50 tx=0x7fcf7002bf60 comp rx=0 tx=0).stop 2023-10-19T10:38:16.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 -- 172.21.15.50:0/2575001342 shutdown_connections 2023-10-19T10:38:16.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/2575001342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf780cfb00 0x7fcf780d0b00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/2575001342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf7815c620 0x7fcf780b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 -- 172.21.15.50:0/2575001342 >> 172.21.15.50:0/2575001342 conn(0x7fcf78071110 msgr2=0x7fcf780714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:16.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 -- 172.21.15.50:0/2575001342 shutdown_connections 2023-10-19T10:38:16.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.650+0000 7fcf7e12d640 1 -- 172.21.15.50:0/2575001342 wait complete. 2023-10-19T10:38:16.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.651+0000 7fcf7e12d640 1 Processor -- start 2023-10-19T10:38:16.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.651+0000 7fcf7e12d640 1 -- start start 2023-10-19T10:38:16.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf7e12d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf780b82f0 0x7fcf780cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf7e12d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf780cfb00 0x7fcf780cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf7e12d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 0x7fcf7814a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.660 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf7e12d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7815f550 con 0x7fcf7815c620 2023-10-19T10:38:16.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf7e12d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78148630 con 0x7fcf780b82f0 2023-10-19T10:38:16.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf7e12d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78148910 con 0x7fcf780cfb00 2023-10-19T10:38:16.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 0x7fcf7814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 0x7fcf7814a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50260/0 (socket says 172.21.15.50:50260) 2023-10-19T10:38:16.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 -- 172.21.15.50:0/3444368444 learned_addr learned my addr 172.21.15.50:0/3444368444 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:16.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf76ffd640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf780cfb00 0x7fcf780cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf777fe640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf780b82f0 0x7fcf780cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 -- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf780cfb00 msgr2=0x7fcf780cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf780cfb00 0x7fcf780cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 -- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf780b82f0 msgr2=0x7fcf780cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:16.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf780b82f0 0x7fcf780cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:16.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.652+0000 7fcf77fff640 1 -- 172.21.15.50:0/3444368444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf70009b30 con 0x7fcf7815c620 2023-10-19T10:38:16.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.653+0000 7fcf77fff640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 0x7fcf7814a570 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009e50 tx=0x7fcf70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.653+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70009510 con 0x7fcf7815c620 2023-10-19T10:38:16.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.653+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7003e450 con 0x7fcf7815c620 2023-10-19T10:38:16.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.653+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7003e800 con 0x7fcf7815c620 2023-10-19T10:38:16.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.653+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf78148bf0 con 0x7fcf7815c620 2023-10-19T10:38:16.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.653+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf780c9690 con 0x7fcf7815c620 2023-10-19T10:38:16.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.654+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf780c9820 con 0x7fcf7815c620 2023-10-19T10:38:16.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.654+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf7003e960 con 0x7fcf7815c620 2023-10-19T10:38:16.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.655+0000 7fcf5ffff640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf6406a080 0x7fcf6406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:16.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.655+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fcf700ab750 con 0x7fcf7815c620 2023-10-19T10:38:16.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.655+0000 7fcf777fe640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf6406a080 0x7fcf6406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:16.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.655+0000 7fcf777fe640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf6406a080 0x7fcf6406c540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:16.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.659+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fcf70076b50 con 0x7fcf7815c620 2023-10-19T10:38:16.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:16.870+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcf781490b0 con 0x7fcf7815c620 2023-10-19T10:38:17.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.527+0000 7fcf5ffff640 1 -- 172.21.15.50:0/3444368444 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf7007a170 con 0x7fcf7815c620 2023-10-19T10:38:17.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf6406a080 msgr2=0x7fcf6406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:17.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf6406a080 0x7fcf6406c540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68008040 comp rx=0 tx=0).stop 2023-10-19T10:38:17.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 msgr2=0x7fcf7814a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:17.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 0x7fcf7814a570 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009e50 tx=0x7fcf70004a40 comp rx=0 tx=0).stop 2023-10-19T10:38:17.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 shutdown_connections 2023-10-19T10:38:17.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcf6406a080 0x7fcf6406c540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf7815c620 0x7fcf7814a570 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf780cfb00 0x7fcf780cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 --2- 172.21.15.50:0/3444368444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf780b82f0 0x7fcf780cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 >> 172.21.15.50:0/3444368444 conn(0x7fcf78071110 msgr2=0x7fcf780b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:17.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 shutdown_connections 2023-10-19T10:38:17.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.530+0000 7fcf7e12d640 1 -- 172.21.15.50:0/3444368444 wait complete. 2023-10-19T10:38:17.547 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-19T10:38:17.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.661+0000 7fc9d940b640 1 Processor -- start 2023-10-19T10:38:17.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.661+0000 7fc9d940b640 1 -- start start 2023-10-19T10:38:17.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d940b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9d415c620 0x7fc9d40b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:17.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d940b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40cfb00 0x7fc9d40d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:17.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d940b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d40cda30 con 0x7fc9d40b82f0 2023-10-19T10:38:17.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d940b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d40ccc10 con 0x7fc9d415c620 2023-10-19T10:38:17.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d940b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4161110 con 0x7fc9d40cfb00 2023-10-19T10:38:17.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d2ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9d415c620 0x7fc9d40b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:17.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9cbfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc9d40b82f0 0x7fc9d40cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39970/0 (socket says 172.21.15.50:39970) 2023-10-19T10:38:17.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9cbfff640 1 -- 172.21.15.50:0/112180258 learned_addr learned my addr 172.21.15.50:0/112180258 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:17.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d37fe640 1 --2- 172.21.15.50:0/112180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40cfb00 0x7fc9d40d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:17.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d2ffd640 1 -- 172.21.15.50:0/112180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40cfb00 msgr2=0x7fc9d40d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:17.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d2ffd640 1 --2- 172.21.15.50:0/112180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40cfb00 0x7fc9d40d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d2ffd640 1 -- 172.21.15.50:0/112180258 >> v1:172.21.15.50:6789/0 conn(0x7fc9d40b82f0 legacy=0x7fc9d40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:17.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d2ffd640 1 -- 172.21.15.50:0/112180258 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d40cce20 con 0x7fc9d415c620 2023-10-19T10:38:17.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.662+0000 7fc9d27fc640 1 -- 172.21.15.50:0/112180258 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106903306 0 0) 0x7fc9d40cda30 con 0x7fc9d40b82f0 2023-10-19T10:38:17.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.663+0000 7fc9d2ffd640 1 --2- 172.21.15.50:0/112180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d40b7db0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40097a0 tx=0x7fc9c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240e5a8f447e029e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:17.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.663+0000 7fc9d27fc640 1 -- 172.21.15.50:0/112180258 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031830 con 0x7fc9d415c620 2023-10-19T10:38:17.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.663+0000 7fc9d27fc640 1 -- 172.21.15.50:0/112180258 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4031990 con 0x7fc9d415c620 2023-10-19T10:38:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.663+0000 7fc9d27fc640 1 -- 172.21.15.50:0/112180258 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031d20 con 0x7fc9d415c620 2023-10-19T10:38:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 -- 172.21.15.50:0/112180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 msgr2=0x7fc9d40b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 --2- 172.21.15.50:0/112180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d40b7db0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40097a0 tx=0x7fc9c402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 -- 172.21.15.50:0/112180258 shutdown_connections 2023-10-19T10:38:17.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 --2- 172.21.15.50:0/112180258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40cfb00 0x7fc9d40d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 --2- 172.21.15.50:0/112180258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d40b7db0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 -- 172.21.15.50:0/112180258 >> 172.21.15.50:0/112180258 conn(0x7fc9d4071110 msgr2=0x7fc9d40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:17.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 -- 172.21.15.50:0/112180258 shutdown_connections 2023-10-19T10:38:17.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.664+0000 7fc9d940b640 1 -- 172.21.15.50:0/112180258 wait complete. 2023-10-19T10:38:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.665+0000 7fc9d940b640 1 Processor -- start 2023-10-19T10:38:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.665+0000 7fc9d940b640 1 -- start start 2023-10-19T10:38:17.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d940b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40b82f0 0x7fc9d40c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:17.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d940b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d40cfb00 0x7fc9d415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:17.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d940b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:17.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d940b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c9010 con 0x7fc9d415c620 2023-10-19T10:38:17.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d940b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d415cf40 con 0x7fc9d40cfb00 2023-10-19T10:38:17.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d940b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d41486c0 con 0x7fc9d40b82f0 2023-10-19T10:38:17.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:17.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42066/0 (socket says 172.21.15.50:42066) 2023-10-19T10:38:17.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 -- 172.21.15.50:0/2554569360 learned_addr learned my addr 172.21.15.50:0/2554569360 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:17.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d2ffd640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40b82f0 0x7fc9d40c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:17.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9cbfff640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d40cfb00 0x7fc9d415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:17.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 -- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40b82f0 msgr2=0x7fc9d40c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:17.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40b82f0 0x7fc9d40c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 -- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d40cfb00 msgr2=0x7fc9d415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:17.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d40cfb00 0x7fc9d415ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:17.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 -- 172.21.15.50:0/2554569360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4009400 con 0x7fc9d415c620 2023-10-19T10:38:17.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9cbfff640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d40cfb00 0x7fc9d415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:17.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9d37fe640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d415f320 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc007e40 tx=0x7fc9cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:17.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.666+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc014620 con 0x7fc9d415c620 2023-10-19T10:38:17.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.667+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc00ee90 con 0x7fc9d415c620 2023-10-19T10:38:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.667+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d41489a0 con 0x7fc9d415c620 2023-10-19T10:38:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.667+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0149e0 con 0x7fc9d415c620 2023-10-19T10:38:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.667+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4148f60 con 0x7fc9d415c620 2023-10-19T10:38:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.667+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d404e420 con 0x7fc9d415c620 2023-10-19T10:38:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.668+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9cc014b40 con 0x7fc9d415c620 2023-10-19T10:38:17.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.668+0000 7fc9cb7fe640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc9b406a050 0x7fc9b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:17.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.668+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fc9cc0893a0 con 0x7fc9d415c620 2023-10-19T10:38:17.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.669+0000 7fc9d2ffd640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc9b406a050 0x7fc9b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:17.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.669+0000 7fc9d2ffd640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc9b406a050 0x7fc9b406c510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4036000 tx=0x7fc9c4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:17.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.673+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc9cc054680 con 0x7fc9d415c620 2023-10-19T10:38:17.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:17.847+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc9d406b6f0 con 0x7fc9d415c620 2023-10-19T10:38:18.536 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.533+0000 7fc9cb7fe640 1 -- 172.21.15.50:0/2554569360 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc9cc057ca0 con 0x7fc9d415c620 2023-10-19T10:38:18.536 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2023-10-19T10:38:18.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc9b406a050 msgr2=0x7fc9b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:18.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc9b406a050 0x7fc9b406c510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4036000 tx=0x7fc9c4008030 comp rx=0 tx=0).stop 2023-10-19T10:38:18.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 msgr2=0x7fc9d415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:18.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d415f320 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc007e40 tx=0x7fc9cc00e4b0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 shutdown_connections 2023-10-19T10:38:18.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc9b406a050 0x7fc9b406c510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9d415c620 0x7fc9d415f320 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d40cfb00 0x7fc9d415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 --2- 172.21.15.50:0/2554569360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9d40b82f0 0x7fc9d40c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.537+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 >> 172.21.15.50:0/2554569360 conn(0x7fc9d4071110 msgr2=0x7fc9d40b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:18.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.538+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 shutdown_connections 2023-10-19T10:38:18.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.538+0000 7fc9d940b640 1 -- 172.21.15.50:0/2554569360 wait complete. 2023-10-19T10:38:18.556 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-19T10:38:18.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 Processor -- start 2023-10-19T10:38:18.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 -- start start 2023-10-19T10:38:18.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f55b40b55c0 0x7f55b40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:18.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:18.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b40cfd80 con 0x7f55b415c620 2023-10-19T10:38:18.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b40ccc10 con 0x7f55b40b55c0 2023-10-19T10:38:18.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.672+0000 7f55bab4e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40cda30 con 0x7f55b40cf930 2023-10-19T10:38:18.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f55b40b55c0 0x7f55b40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:18.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f55b40b55c0 0x7f55b40cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42068/0 (socket says 172.21.15.50:42068) 2023-10-19T10:38:18.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 -- 172.21.15.50:0/2116534770 learned_addr learned my addr 172.21.15.50:0/2116534770 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:18.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b88ea640 1 --1- 172.21.15.50:0/2116534770 >> v1:172.21.15.50:6789/0 conn(0x7f55b415c620 0x7f55b40b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39974/0 (socket says 172.21.15.50:39974) 2023-10-19T10:38:18.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b90eb640 1 --2- 172.21.15.50:0/2116534770 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:18.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 -- 172.21.15.50:0/2116534770 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 msgr2=0x7f55b40d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:18.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 --2- 172.21.15.50:0/2116534770 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 -- 172.21.15.50:0/2116534770 >> v1:172.21.15.50:6789/0 conn(0x7f55b415c620 legacy=0x7f55b40b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:18.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b3fff640 1 -- 172.21.15.50:0/2116534770 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4161110 con 0x7f55b40b55c0 2023-10-19T10:38:18.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.673+0000 7f55b37fe640 1 -- 172.21.15.50:0/2116534770 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517224891 0 0) 0x7f55a8001110 con 0x7f55b415c620 2023-10-19T10:38:18.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55b3fff640 1 --2- 172.21.15.50:0/2116534770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b40cf3f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009870 tx=0x7f55a002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea39744f1b23ce3a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:18.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55b37fe640 1 -- 172.21.15.50:0/2116534770 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0030840 con 0x7f55b40b55c0 2023-10-19T10:38:18.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55b37fe640 1 -- 172.21.15.50:0/2116534770 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a00309a0 con 0x7f55b40b55c0 2023-10-19T10:38:18.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55b37fe640 1 -- 172.21.15.50:0/2116534770 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0030d30 con 0x7f55b40b55c0 2023-10-19T10:38:18.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 -- 172.21.15.50:0/2116534770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 msgr2=0x7f55b40cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:18.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 --2- 172.21.15.50:0/2116534770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b40cf3f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009870 tx=0x7f55a002b980 comp rx=0 tx=0).stop 2023-10-19T10:38:18.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 -- 172.21.15.50:0/2116534770 shutdown_connections 2023-10-19T10:38:18.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 --2- 172.21.15.50:0/2116534770 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 --2- 172.21.15.50:0/2116534770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b40cf3f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 -- 172.21.15.50:0/2116534770 >> 172.21.15.50:0/2116534770 conn(0x7f55b4071110 msgr2=0x7f55b40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:18.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 -- 172.21.15.50:0/2116534770 shutdown_connections 2023-10-19T10:38:18.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.674+0000 7f55bab4e640 1 -- 172.21.15.50:0/2116534770 wait complete. 2023-10-19T10:38:18.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 Processor -- start 2023-10-19T10:38:18.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 -- start start 2023-10-19T10:38:18.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:18.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:18.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55b415c620 0x7f55b40a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:18.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40aa6f0 con 0x7f55b40b55c0 2023-10-19T10:38:18.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.676+0000 7f55bab4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b414cba0 con 0x7f55b415c620 2023-10-19T10:38:18.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55bab4e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40a9a70 con 0x7f55b40cf930 2023-10-19T10:38:18.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:18.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b414c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42080/0 (socket says 172.21.15.50:42080) 2023-10-19T10:38:18.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 -- 172.21.15.50:0/524772068 learned_addr learned my addr 172.21.15.50:0/524772068 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:18.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b3fff640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:18.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 -- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 msgr2=0x7f55b40abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:18.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b90eb640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55b415c620 0x7f55b40a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:18.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 -- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55b415c620 msgr2=0x7f55b40a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:18.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55b415c620 0x7f55b40a8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:18.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 -- 172.21.15.50:0/524772068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a00094d0 con 0x7f55b40b55c0 2023-10-19T10:38:18.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b3fff640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:18.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b90eb640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55b415c620 0x7f55b40a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:18.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b88ea640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b414c660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f55a800c430 tx=0x7f55a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:18.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80074e0 con 0x7f55b40b55c0 2023-10-19T10:38:18.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a8007660 con 0x7f55b40b55c0 2023-10-19T10:38:18.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b40a9d50 con 0x7f55b40b55c0 2023-10-19T10:38:18.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8004a20 con 0x7f55b40b55c0 2023-10-19T10:38:18.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.677+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b40aa310 con 0x7f55b40b55c0 2023-10-19T10:38:18.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.678+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b404e420 con 0x7f55b40b55c0 2023-10-19T10:38:18.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.679+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f55a8004b80 con 0x7f55b40b55c0 2023-10-19T10:38:18.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.679+0000 7f55b17fa640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f559806a080 0x7f559806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:18.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.679+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f55a80883f0 con 0x7f55b40b55c0 2023-10-19T10:38:18.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.680+0000 7f55b3fff640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f559806a080 0x7f559806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:18.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.680+0000 7f55b3fff640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f559806a080 0x7f559806c540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55a0036000 tx=0x7f55a003b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:18.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.684+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f55a8053900 con 0x7f55b40b55c0 2023-10-19T10:38:18.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:18.863+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f55b40cfd80 con 0x7f55b40b55c0 2023-10-19T10:38:19.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.539+0000 7f55b17fa640 1 -- 172.21.15.50:0/524772068 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 169+0+0 (secure 0 0 0) 0x7f55b40cfd80 con 0x7f55b40b55c0 2023-10-19T10:38:19.542 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2023-10-19T10:38:19.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.544+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f559806a080 msgr2=0x7f559806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:19.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.544+0000 7f55bab4e640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f559806a080 0x7f559806c540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55a0036000 tx=0x7f55a003b040 comp rx=0 tx=0).stop 2023-10-19T10:38:19.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.544+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 msgr2=0x7f55b414c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:19.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.544+0000 7f55bab4e640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b414c660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f55a800c430 tx=0x7f55a800c8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 shutdown_connections 2023-10-19T10:38:19.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f559806a080 0x7f559806c540 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55b415c620 0x7f55b40a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55b40cf930 0x7f55b40abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 --2- 172.21.15.50:0/524772068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55b40b55c0 0x7f55b414c660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 >> 172.21.15.50:0/524772068 conn(0x7f55b4071110 msgr2=0x7f55b40714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:19.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 shutdown_connections 2023-10-19T10:38:19.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.545+0000 7f55bab4e640 1 -- 172.21.15.50:0/524772068 wait complete. 2023-10-19T10:38:19.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-19T10:38:19.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.677+0000 7ff0f848b640 1 Processor -- start 2023-10-19T10:38:19.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.677+0000 7ff0f848b640 1 -- start start 2023-10-19T10:38:19.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.678+0000 7ff0f848b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 0x7ff0f00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.678+0000 7ff0f848b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.678+0000 7ff0f848b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f0155170 con 0x7ff0f015e2a0 2023-10-19T10:38:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.678+0000 7ff0f848b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f015acb0 con 0x7ff0f0154d90 2023-10-19T10:38:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.678+0000 7ff0f848b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f015efd0 con 0x7ff0f015b980 2023-10-19T10:38:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f6227640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 0x7ff0f00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f5a26640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f6227640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 0x7ff0f00b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:54062/0 (socket says 172.21.15.50:54062) 2023-10-19T10:38:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f5a26640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:51106/0 (socket says 172.21.15.50:51106) 2023-10-19T10:38:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f6227640 1 -- 172.21.15.50:0/1647206768 learned_addr learned my addr 172.21.15.50:0/1647206768 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f6a28640 1 --1- 172.21.15.50:0/1647206768 >> v1:172.21.15.183:6789/0 conn(0x7ff0f015e2a0 0x7ff0f015e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:50186/0 (socket says 172.21.15.50:50186) 2023-10-19T10:38:19.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f5a26640 1 -- 172.21.15.50:0/1647206768 >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 msgr2=0x7ff0f00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:19.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f5a26640 1 --2- 172.21.15.50:0/1647206768 >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 0x7ff0f00b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f5a26640 1 -- 172.21.15.50:0/1647206768 >> v1:172.21.15.183:6789/0 conn(0x7ff0f015e2a0 legacy=0x7ff0f015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:19.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f5a26640 1 -- 172.21.15.50:0/1647206768 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f015aea0 con 0x7ff0f015b980 2023-10-19T10:38:19.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.679+0000 7ff0f6227640 1 --2- 172.21.15.50:0/1647206768 >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 0x7ff0f00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:19.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.680+0000 7ff0f5a26640 1 --2- 172.21.15.50:0/1647206768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00b22a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40097a0 tx=0x7ff0e402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b60b1e9e4ff8ff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:19.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.680+0000 7ff0f5225640 1 -- 172.21.15.50:0/1647206768 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4031830 con 0x7ff0f015b980 2023-10-19T10:38:19.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.680+0000 7ff0f5225640 1 -- 172.21.15.50:0/1647206768 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e4031990 con 0x7ff0f015b980 2023-10-19T10:38:19.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.680+0000 7ff0f5225640 1 -- 172.21.15.50:0/1647206768 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4031d20 con 0x7ff0f015b980 2023-10-19T10:38:19.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 -- 172.21.15.50:0/1647206768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 msgr2=0x7ff0f00b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:19.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 --2- 172.21.15.50:0/1647206768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00b22a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40097a0 tx=0x7ff0e402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.688 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 -- 172.21.15.50:0/1647206768 shutdown_connections 2023-10-19T10:38:19.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 --2- 172.21.15.50:0/1647206768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00b22a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 --2- 172.21.15.50:0/1647206768 >> v2:172.21.15.183:3300/0 conn(0x7ff0f0154d90 0x7ff0f00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 -- 172.21.15.50:0/1647206768 >> 172.21.15.50:0/1647206768 conn(0x7ff0f006f440 msgr2=0x7ff0f006f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:19.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 -- 172.21.15.50:0/1647206768 shutdown_connections 2023-10-19T10:38:19.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.681+0000 7ff0f848b640 1 -- 172.21.15.50:0/1647206768 wait complete. 2023-10-19T10:38:19.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.682+0000 7ff0f848b640 1 Processor -- start 2023-10-19T10:38:19.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.682+0000 7ff0f848b640 1 -- start start 2023-10-19T10:38:19.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f848b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0154d90 0x7ff0f00c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:19.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f848b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00d0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:19.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f848b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 0x7ff0f01e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:19.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f848b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00cf8e0 con 0x7ff0f015e2a0 2023-10-19T10:38:19.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f848b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00c2f00 con 0x7ff0f0154d90 2023-10-19T10:38:19.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f848b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00cecd0 con 0x7ff0f015b980 2023-10-19T10:38:19.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 0x7ff0f01e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:19.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 0x7ff0f01e5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42092/0 (socket says 172.21.15.50:42092) 2023-10-19T10:38:19.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 -- 172.21.15.50:0/2459445823 learned_addr learned my addr 172.21.15.50:0/2459445823 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:19.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6227640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0154d90 0x7ff0f00c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:19.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f5a26640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:19.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 -- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 msgr2=0x7ff0f00d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:19.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 -- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0154d90 msgr2=0x7ff0f00c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:19.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0154d90 0x7ff0f00c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:19.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 -- 172.21.15.50:0/2459445823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80094d0 con 0x7ff0f015e2a0 2023-10-19T10:38:19.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.683+0000 7ff0f6a28640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 0x7ff0f01e5580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c730 tx=0x7ff0e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:19.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.684+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000ce40 con 0x7ff0f015e2a0 2023-10-19T10:38:19.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.684+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e000a640 con 0x7ff0f015e2a0 2023-10-19T10:38:19.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.684+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e4009400 con 0x7ff0f015e2a0 2023-10-19T10:38:19.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.684+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000a9f0 con 0x7ff0f015e2a0 2023-10-19T10:38:19.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.684+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f00cf290 con 0x7ff0f015e2a0 2023-10-19T10:38:19.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.684+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f00cfa50 con 0x7ff0f015e2a0 2023-10-19T10:38:19.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.685+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff0e000ab50 con 0x7ff0f015e2a0 2023-10-19T10:38:19.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.686+0000 7ff0eeffd640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff0dc06a080 0x7ff0dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:19.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.686+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6047+0+0 (secure 0 0 0) 0x7ff0e0087bf0 con 0x7ff0f015e2a0 2023-10-19T10:38:19.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.686+0000 7ff0f6227640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff0dc06a080 0x7ff0dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:19.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.686+0000 7ff0f6227640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff0dc06a080 0x7ff0dc06c540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff0f014da10 tx=0x7ff0e8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:19.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.690+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff0e00533a0 con 0x7ff0f015e2a0 2023-10-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:19.868+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0f004e420 con 0x7ff0f015e2a0 2023-10-19T10:38:20.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.543+0000 7ff0eeffd640 1 -- 172.21.15.50:0/2459445823 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff0e00170d0 con 0x7ff0f015e2a0 2023-10-19T10:38:20.548 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' removed 2023-10-19T10:38:20.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.547+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff0dc06a080 msgr2=0x7ff0dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.547+0000 7ff0f848b640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff0dc06a080 0x7ff0dc06c540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff0f014da10 tx=0x7ff0e8009240 comp rx=0 tx=0).stop 2023-10-19T10:38:20.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.547+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 msgr2=0x7ff0f01e5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.547+0000 7ff0f848b640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 0x7ff0f01e5580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c730 tx=0x7ff0e000cbf0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.548+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 shutdown_connections 2023-10-19T10:38:20.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.548+0000 7ff0f848b640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff0dc06a080 0x7ff0dc06c540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.548+0000 7ff0f848b640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f015e2a0 0x7ff0f01e5580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.548+0000 7ff0f848b640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff0f015b980 0x7ff0f00d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.548+0000 7ff0f848b640 1 --2- 172.21.15.50:0/2459445823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f0154d90 0x7ff0f00c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.548+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 >> 172.21.15.50:0/2459445823 conn(0x7ff0f006f440 msgr2=0x7ff0f0152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:20.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.549+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 shutdown_connections 2023-10-19T10:38:20.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.549+0000 7ff0f848b640 1 -- 172.21.15.50:0/2459445823 wait complete. 2023-10-19T10:38:20.566 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-19T10:38:20.566 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-19T10:38:20.567 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-19T10:38:20.567 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:20.567 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-19T10:38:20.568 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-19T10:38:20.568 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-19T10:38:20.568 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:20.568 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-19T10:38:20.569 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-19T10:38:20.569 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-19T10:38:20.569 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:20.570 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-19T10:38:20.570 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-19T10:38:20.570 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-19T10:38:20.570 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:38:20.577 INFO:tasks.ceph.mds.a:Stopped 2023-10-19T10:38:20.578 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-19T10:38:20.579 INFO:tasks.ceph.mds.c:Stopped 2023-10-19T10:38:20.579 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-19T10:38:20.581 INFO:tasks.ceph.mds.b:Stopped 2023-10-19T10:38:20.581 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-19T10:38:20.582 INFO:tasks.ceph.mds.d:Stopped 2023-10-19T10:38:20.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-19T10:38:20.621 INFO:tasks.ceph.mds.a:Started 2023-10-19T10:38:20.622 DEBUG:teuthology.parallel:result is None 2023-10-19T10:38:20.623 INFO:tasks.ceph.mds.c:Started 2023-10-19T10:38:20.624 DEBUG:teuthology.parallel:result is None 2023-10-19T10:38:20.624 INFO:tasks.ceph.mds.b:Started 2023-10-19T10:38:20.625 DEBUG:teuthology.parallel:result is None 2023-10-19T10:38:20.625 INFO:tasks.ceph.mds.d:Started 2023-10-19T10:38:20.626 DEBUG:teuthology.parallel:result is None 2023-10-19T10:38:20.626 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-19T10:38:20.706 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-10-19T10:38:20.707 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-10-19T10:38:20.709 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2023-10-19T10:38:20.710 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2023-10-19T10:38:20.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.741+0000 7fabfdf40640 1 Processor -- start 2023-10-19T10:38:20.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.741+0000 7fabfdf40640 1 -- start start 2023-10-19T10:38:20.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabfdf40640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fabf8069f10 0x7fabf815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabfdf40640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf815f050 con 0x7fabf80c27d0 2023-10-19T10:38:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabfdf40640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf815e0c0 con 0x7fabf815b780 2023-10-19T10:38:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabfdf40640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80cb990 con 0x7fabf8069f10 2023-10-19T10:38:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabf6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fabf8069f10 0x7fabf815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:20.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabf7fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fabf80c27d0 0x7fabf8070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39982/0 (socket says 172.21.15.50:39982) 2023-10-19T10:38:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.742+0000 7fabf7fff640 1 -- 172.21.15.50:0/2769878326 learned_addr learned my addr 172.21.15.50:0/2769878326 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.743+0000 7fabf67fc640 1 -- 172.21.15.50:0/2769878326 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952735470 0 0) 0x7fabf0001490 con 0x7fabf80c27d0 2023-10-19T10:38:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.743+0000 7fabf67fc640 1 -- 172.21.15.50:0/2769878326 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe0003520 con 0x7fabf80c27d0 2023-10-19T10:38:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.743+0000 7fabf6ffd640 1 -- 172.21.15.50:0/2769878326 >> v1:172.21.15.183:6789/0 conn(0x7fabf815b780 legacy=0x7fabf8069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.743+0000 7fabf6ffd640 1 -- 172.21.15.50:0/2769878326 >> v1:172.21.15.50:6789/0 conn(0x7fabf80c27d0 legacy=0x7fabf8070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.743+0000 7fabf6ffd640 1 -- 172.21.15.50:0/2769878326 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80cc850 con 0x7fabf8069f10 2023-10-19T10:38:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabf6ffd640 1 --2- 172.21.15.50:0/2769878326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf8069f10 0x7fabf815eb10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097a0 tx=0x7fabec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a991496d5c94fd47 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabf67fc640 1 -- 172.21.15.50:0/2769878326 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec02e070 con 0x7fabf8069f10 2023-10-19T10:38:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabf67fc640 1 -- 172.21.15.50:0/2769878326 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec0315a0 con 0x7fabf8069f10 2023-10-19T10:38:20.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabf67fc640 1 -- 172.21.15.50:0/2769878326 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec031910 con 0x7fabf8069f10 2023-10-19T10:38:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 -- 172.21.15.50:0/2769878326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf8069f10 msgr2=0x7fabf815eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 --2- 172.21.15.50:0/2769878326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf8069f10 0x7fabf815eb10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097a0 tx=0x7fabec02b840 comp rx=0 tx=0).stop 2023-10-19T10:38:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 -- 172.21.15.50:0/2769878326 shutdown_connections 2023-10-19T10:38:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 --2- 172.21.15.50:0/2769878326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf8069f10 0x7fabf815eb10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 -- 172.21.15.50:0/2769878326 >> 172.21.15.50:0/2769878326 conn(0x7fabf80cbbc0 msgr2=0x7fabf80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 -- 172.21.15.50:0/2769878326 shutdown_connections 2023-10-19T10:38:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.744+0000 7fabfdf40640 1 -- 172.21.15.50:0/2769878326 wait complete. 2023-10-19T10:38:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 Processor -- start 2023-10-19T10:38:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 -- start start 2023-10-19T10:38:20.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 0x7fabf8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 0x7fabf80b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 0x7fabf80b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80b2f50 con 0x7fabf80c27d0 2023-10-19T10:38:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.746+0000 7fabfdf40640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80b2ac0 con 0x7fabf815b780 2023-10-19T10:38:20.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabfdf40640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80b1ef0 con 0x7fabf8069f10 2023-10-19T10:38:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 0x7fabf80b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 0x7fabf80b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf77fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 0x7fabf8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf77fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 0x7fabf8157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:51148/0 (socket says 172.21.15.50:51148) 2023-10-19T10:38:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 0x7fabf80b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42140/0 (socket says 172.21.15.50:42140) 2023-10-19T10:38:20.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 0x7fabf80b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:54124/0 (socket says 172.21.15.50:54124) 2023-10-19T10:38:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf77fe640 1 -- 172.21.15.50:0/1865270221 learned_addr learned my addr 172.21.15.50:0/1865270221 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 -- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 msgr2=0x7fabf8157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 0x7fabf8157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 -- 172.21.15.50:0/1865270221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 msgr2=0x7fabf80b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:20.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 0x7fabf80b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 -- 172.21.15.50:0/1865270221 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec009400 con 0x7fabf815b780 2023-10-19T10:38:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf77fe640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 0x7fabf8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf6ffd640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 0x7fabf80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabf7fff640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 0x7fabf80b4050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabf000caa0 tx=0x7fabf000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:20.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.747+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000ac00 con 0x7fabf815b780 2023-10-19T10:38:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.748+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000ad60 con 0x7fabf815b780 2023-10-19T10:38:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.748+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf80b2190 con 0x7fabf815b780 2023-10-19T10:38:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.748+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0012600 con 0x7fabf815b780 2023-10-19T10:38:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.748+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf80b2750 con 0x7fabf815b780 2023-10-19T10:38:20.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.749+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf8071c70 con 0x7fabf815b780 2023-10-19T10:38:20.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.749+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fabf0007070 con 0x7fabf815b780 2023-10-19T10:38:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.749+0000 7fabdffff640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fabe406a050 0x7fabe406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.750+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fabf00880a0 con 0x7fabf815b780 2023-10-19T10:38:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.750+0000 7fabf77fe640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fabe406a050 0x7fabe406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.750+0000 7fabf77fe640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fabe406a050 0x7fabe406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabe8001b90 tx=0x7fabe8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:20.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.754+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fabf0053a90 con 0x7fabf815b780 2023-10-19T10:38:20.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:20.944+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fabf815f050 con 0x7fabf815b780 2023-10-19T10:38:22.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.124+0000 7fabdffff640 1 -- 172.21.15.50:0/1865270221 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7fabf815f050 con 0x7fabf815b780 2023-10-19T10:38:22.127 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2023-10-19T10:38:22.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fabe406a050 msgr2=0x7fabe406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fabe406a050 0x7fabe406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabe8001b90 tx=0x7fabe8009040 comp rx=0 tx=0).stop 2023-10-19T10:38:22.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 msgr2=0x7fabf80b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 0x7fabf80b4050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabf000caa0 tx=0x7fabf000cf60 comp rx=0 tx=0).stop 2023-10-19T10:38:22.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 shutdown_connections 2023-10-19T10:38:22.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fabe406a050 0x7fabe406c510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabf815b780 0x7fabf80b4050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabf80c27d0 0x7fabf80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 --2- 172.21.15.50:0/1865270221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fabf8069f10 0x7fabf8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 >> 172.21.15.50:0/1865270221 conn(0x7fabf80cbbc0 msgr2=0x7fabf815d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:22.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 shutdown_connections 2023-10-19T10:38:22.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.127+0000 7fabfdf40640 1 -- 172.21.15.50:0/1865270221 wait complete. 2023-10-19T10:38:22.143 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-19T10:38:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.253+0000 7ff23bcb1640 1 Processor -- start 2023-10-19T10:38:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.253+0000 7ff23bcb1640 1 -- start start 2023-10-19T10:38:22.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23bcb1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2340caf90 0x7ff234159210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23bcb1640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2340c95d0 con 0x7ff234159750 2023-10-19T10:38:22.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23bcb1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2340c8ca0 con 0x7ff234156510 2023-10-19T10:38:22.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23bcb1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23406f770 con 0x7ff2340caf90 2023-10-19T10:38:22.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23a24e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff234159750 0x7ff2340cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39996/0 (socket says 172.21.15.50:39996) 2023-10-19T10:38:22.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23a24e640 1 -- 172.21.15.50:0/2227458757 learned_addr learned my addr 172.21.15.50:0/2227458757 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:22.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff23924c640 1 --2- 172.21.15.50:0/2227458757 >> v2:172.21.15.183:3300/0 conn(0x7ff2340caf90 0x7ff234159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893748218 0 0) 0x7ff2340c95d0 con 0x7ff234159750 2023-10-19T10:38:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff21c003520 con 0x7ff234159750 2023-10-19T10:38:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376334304 0 0) 0x7ff2340c8ca0 con 0x7ff234156510 2023-10-19T10:38:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2340c95d0 con 0x7ff234156510 2023-10-19T10:38:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3836618608 0 0) 0x7ff21c003520 con 0x7ff234159750 2023-10-19T10:38:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 >> v2:172.21.15.183:3300/0 conn(0x7ff2340caf90 msgr2=0x7ff234159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:22.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 --2- 172.21.15.50:0/2227458757 >> v2:172.21.15.183:3300/0 conn(0x7ff2340caf90 0x7ff234159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 >> v1:172.21.15.183:6789/0 conn(0x7ff234156510 legacy=0x7ff2340ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340c8f80 con 0x7ff234159750 2023-10-19T10:38:22.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7ff2240019f0 con 0x7ff234159750 2023-10-19T10:38:22.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 >> v1:172.21.15.50:6789/0 conn(0x7ff234159750 legacy=0x7ff2340cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c003b00 0x7ff21c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff21c0047c0 0x7ff21c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c005180 0x7ff21c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c00bfd0 con 0x7ff21c005180 2023-10-19T10:38:22.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c00c140 con 0x7ff21c003b00 2023-10-19T10:38:22.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.254+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c00c300 con 0x7ff21c0047c0 2023-10-19T10:38:22.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23924c640 1 --2- 172.21.15.50:0/2227458757 >> v2:172.21.15.183:3300/0 conn(0x7ff2340caf90 0x7ff234159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:22.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23a24e640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c005180 0x7ff21c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff239a4d640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c003b00 0x7ff21c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23924c640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff21c0047c0 0x7ff21c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23a24e640 1 -- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff21c0047c0 msgr2=0x7ff21c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:22.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23a24e640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff21c0047c0 0x7ff21c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23a24e640 1 -- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c003b00 msgr2=0x7ff21c003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:22.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23a24e640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c003b00 0x7ff21c003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.255+0000 7ff23a24e640 1 -- 172.21.15.50:0/2227458757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c003520 con 0x7ff21c005180 2023-10-19T10:38:22.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23a24e640 1 -- 172.21.15.50:0/2227458757 reap_dead start 2023-10-19T10:38:22.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23a24e640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c005180 0x7ff21c00b980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff22400a7a0 tx=0x7ff21c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41237073f6cae54c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:22.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224034630 con 0x7ff21c005180 2023-10-19T10:38:22.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22402f070 con 0x7ff21c005180 2023-10-19T10:38:22.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff238a4b640 1 -- 172.21.15.50:0/2227458757 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240349b0 con 0x7ff21c005180 2023-10-19T10:38:22.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 -- 172.21.15.50:0/2227458757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c005180 msgr2=0x7ff21c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c005180 0x7ff21c00b980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff22400a7a0 tx=0x7ff21c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 -- 172.21.15.50:0/2227458757 shutdown_connections 2023-10-19T10:38:22.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/2227458757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c005180 0x7ff21c00b980 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 -- 172.21.15.50:0/2227458757 >> 172.21.15.50:0/2227458757 conn(0x7ff234069e90 msgr2=0x7ff2340afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:22.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 -- 172.21.15.50:0/2227458757 shutdown_connections 2023-10-19T10:38:22.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.256+0000 7ff23bcb1640 1 -- 172.21.15.50:0/2227458757 wait complete. 2023-10-19T10:38:22.270 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 Processor -- start 2023-10-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 -- start start 2023-10-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c005180 0x7ff234154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2340c31c0 0x7ff2340b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 0x7ff2341f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c4310 con 0x7ff2340c3640 2023-10-19T10:38:22.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c45b0 con 0x7ff21c005180 2023-10-19T10:38:22.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23bcb1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c4890 con 0x7ff2340c31c0 2023-10-19T10:38:22.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23a24e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 0x7ff2341f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23a24e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 0x7ff2341f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42144/0 (socket says 172.21.15.50:42144) 2023-10-19T10:38:22.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.258+0000 7ff23a24e640 1 -- 172.21.15.50:0/532409696 learned_addr learned my addr 172.21.15.50:0/532409696 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff239a4d640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c005180 0x7ff234154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23924c640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2340c31c0 0x7ff2340b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23a24e640 1 -- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2340c31c0 msgr2=0x7ff2340b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23a24e640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2340c31c0 0x7ff2340b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23a24e640 1 -- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c005180 msgr2=0x7ff234154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23a24e640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c005180 0x7ff234154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23a24e640 1 -- 172.21.15.50:0/532409696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340c29c0 con 0x7ff2340c3640 2023-10-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff239a4d640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c005180 0x7ff234154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23a24e640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 0x7ff2341f07d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff224002230 tx=0x7ff224001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224004d60 con 0x7ff2340c3640 2023-10-19T10:38:22.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22403f470 con 0x7ff2340c3640 2023-10-19T10:38:22.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22403f800 con 0x7ff2340c3640 2023-10-19T10:38:22.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2340c2ca0 con 0x7ff2340c3640 2023-10-19T10:38:22.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.259+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2341f14d0 con 0x7ff2340c3640 2023-10-19T10:38:22.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.260+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2340c2820 con 0x7ff2340c3640 2023-10-19T10:38:22.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.261+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff224031030 con 0x7ff2340c3640 2023-10-19T10:38:22.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.261+0000 7ff2327fc640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff22006a050 0x7ff22006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.261+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff2240a9720 con 0x7ff2340c3640 2023-10-19T10:38:22.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.261+0000 7ff239a4d640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff22006a050 0x7ff22006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.276 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.262+0000 7ff239a4d640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff22006a050 0x7ff22006c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff2340c85f0 tx=0x7ff22c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:22.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.266+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff2340c2820 con 0x7ff2340c3640 2023-10-19T10:38:22.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.506+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff2340c2820 con 0x7ff2340c3640 2023-10-19T10:38:22.509 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.506+0000 7ff2327fc640 1 -- 172.21.15.50:0/532409696 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff2340c2820 con 0x7ff2340c3640 2023-10-19T10:38:22.509 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:22.510 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:38:22.510 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2023-10-19T10:38:22.510 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.510 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2023-10-19T10:38:22.510 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlRrqKERAA4Ap/UVvrMlJVVcCkt0gd6Q==", 2023-10-19T10:38:22.511 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.511 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.511 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2023-10-19T10:38:22.511 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2023-10-19T10:38:22.511 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.512 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.512 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.512 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.512 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2023-10-19T10:38:22.512 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlVUq6HhAAxkq93v09PLVHLcq0RgMvYA==", 2023-10-19T10:38:22.513 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.513 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.513 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2023-10-19T10:38:22.513 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2023-10-19T10:38:22.513 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.514 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.514 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.514 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2023-10-19T10:38:22.514 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlg+qDGRAAfdK7WLKio8hmPYJkWib0Nw==", 2023-10-19T10:38:22.515 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.515 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.515 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2023-10-19T10:38:22.515 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2023-10-19T10:38:22.515 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.516 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.516 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.516 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2023-10-19T10:38:22.516 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlxk4cJhAAAf3p7z1wG6pB0TYpyKoEmg==", 2023-10-19T10:38:22.516 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.517 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.517 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2023-10-19T10:38:22.517 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2023-10-19T10:38:22.517 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.517 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.518 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.518 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2023-10-19T10:38:22.518 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQBjFlvGmmExAAC5CLzIPkJo/EBVkpnX4ECQ==", 2023-10-19T10:38:22.518 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.519 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.519 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.519 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.519 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.519 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.520 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.520 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2023-10-19T10:38:22.520 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCTBjFlIkp4EhAACncwsFhJPpqc8svxntHDLg==", 2023-10-19T10:38:22.520 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.520 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.521 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.521 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.521 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.521 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.522 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2023-10-19T10:38:22.522 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCWBjFl06cYEhAA/rxMv0AQfiRoZJkquKcaTQ==", 2023-10-19T10:38:22.522 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.522 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.522 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.523 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.523 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.523 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.523 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.523 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2023-10-19T10:38:22.524 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCZBjFl0/TUEBAAWoTV+/BBZVx4BIa5Bz57aw==", 2023-10-19T10:38:22.524 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.524 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.524 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.524 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.525 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.525 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.525 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2023-10-19T10:38:22.525 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCfBjFlA3/OGRAA3aeeufXss+eAcAxoT8scOw==", 2023-10-19T10:38:22.525 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.526 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.526 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.526 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.526 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.526 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.527 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.527 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2023-10-19T10:38:22.527 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCiBjFl5y8KGBAADp0V5u40szv14/1Jhlco/A==", 2023-10-19T10:38:22.527 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.527 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.528 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.528 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.528 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.528 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2023-10-19T10:38:22.529 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQClBjFljPz4FRAAQfXMoK9aLXBsjX60h+cNDQ==", 2023-10-19T10:38:22.529 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.529 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.529 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.530 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.530 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.530 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.530 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.530 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2023-10-19T10:38:22.531 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCoBjFljvFbFBAAOUga0rBE6sm1IHvuQzi7Ww==", 2023-10-19T10:38:22.531 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.531 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2023-10-19T10:38:22.531 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2023-10-19T10:38:22.531 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.532 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.532 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.532 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2023-10-19T10:38:22.532 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlR3SvKhAAQGRIzy2kDJzeLJAJbvCD9Q==", 2023-10-19T10:38:22.532 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.533 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.533 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2023-10-19T10:38:22.533 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2023-10-19T10:38:22.533 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2023-10-19T10:38:22.533 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.534 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.534 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2023-10-19T10:38:22.534 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlHmwhMhAAxbIws9tQaedL+MvOUYfljA==", 2023-10-19T10:38:22.535 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.535 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.535 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2023-10-19T10:38:22.535 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2023-10-19T10:38:22.535 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2023-10-19T10:38:22.536 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.536 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.536 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.536 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2023-10-19T10:38:22.536 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlJyiOLhAAosLYQg1Va20wfFzst/qkAA==", 2023-10-19T10:38:22.536 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.537 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.537 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2023-10-19T10:38:22.537 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2023-10-19T10:38:22.537 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2023-10-19T10:38:22.537 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.538 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.538 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.538 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2023-10-19T10:38:22.538 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlgb7iNxAAIkDDvbzY35oqbjo2R6duaA==", 2023-10-19T10:38:22.538 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.539 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2023-10-19T10:38:22.539 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2023-10-19T10:38:22.539 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2023-10-19T10:38:22.539 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2023-10-19T10:38:22.539 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.540 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.540 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2023-10-19T10:38:22.540 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCNBjFlrvLsLhAAwy3PiELjx2uPhn6Skrknuw==", 2023-10-19T10:38:22.540 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.541 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2023-10-19T10:38:22.541 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2023-10-19T10:38:22.541 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2023-10-19T10:38:22.541 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.541 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.541 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.542 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.542 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-mds", 2023-10-19T10:38:22.542 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCuBjFlVlEsFBAA/EnkJW/tXBeasyca89AnYQ==", 2023-10-19T10:38:22.542 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.542 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-mds" 2023-10-19T10:38:22.543 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.543 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.543 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-mgr", 2023-10-19T10:38:22.543 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCuBjFla1osFBAAsTiiO4E8ZTZtYiWyvYw72g==", 2023-10-19T10:38:22.544 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.544 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-19T10:38:22.544 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.544 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.544 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.545 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-osd", 2023-10-19T10:38:22.545 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCuBjFlxmEsFBAAy464N5ClkRvegDX6LdudzA==", 2023-10-19T10:38:22.545 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.545 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-osd" 2023-10-19T10:38:22.545 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.546 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.546 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rbd", 2023-10-19T10:38:22.546 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCuBjFlG2ksFBAA+KxwdNKZfWWW8lmQjqJzRw==", 2023-10-19T10:38:22.546 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.546 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-19T10:38:22.547 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.547 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.547 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-19T10:38:22.547 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCuBjFld3AsFBAAHH8Lvrw1Gf07hNui5C6r5w==", 2023-10-19T10:38:22.548 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.548 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-19T10:38:22.548 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.548 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.549 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rgw", 2023-10-19T10:38:22.549 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCuBjFl2HcsFBAAVI1J8Q7RskjGDVHFR3QNWw==", 2023-10-19T10:38:22.549 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.549 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-19T10:38:22.549 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.550 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.550 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2023-10-19T10:38:22.550 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlXJGxDRAAGXHVR7kbvvZrRnoXOw18kg==", 2023-10-19T10:38:22.550 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.550 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2023-10-19T10:38:22.551 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2023-10-19T10:38:22.551 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.551 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:22.551 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:22.552 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2023-10-19T10:38:22.552 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCOBjFlv7p8CRAAg8SiGH5E57/O6+UlWpgoCw==", 2023-10-19T10:38:22.552 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2023-10-19T10:38:22.552 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2023-10-19T10:38:22.552 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2023-10-19T10:38:22.553 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2023-10-19T10:38:22.553 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.553 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:22.553 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2023-10-19T10:38:22.553 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:38:22.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff22006a050 msgr2=0x7ff22006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff22006a050 0x7ff22006c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff2340c85f0 tx=0x7ff22c009040 comp rx=0 tx=0).stop 2023-10-19T10:38:22.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 msgr2=0x7ff2341f07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 0x7ff2341f07d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff224002230 tx=0x7ff224001d00 comp rx=0 tx=0).stop 2023-10-19T10:38:22.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 shutdown_connections 2023-10-19T10:38:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff22006a050 0x7ff22006c510 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2340c3640 0x7ff2341f07d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2340c31c0 0x7ff2340b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 --2- 172.21.15.50:0/532409696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c005180 0x7ff234154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 >> 172.21.15.50:0/532409696 conn(0x7ff234069e90 msgr2=0x7ff2340ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:22.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 shutdown_connections 2023-10-19T10:38:22.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.508+0000 7ff23bcb1640 1 -- 172.21.15.50:0/532409696 wait complete. 2023-10-19T10:38:22.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-19T10:38:22.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.664+0000 7f2331c12640 1 Processor -- start 2023-10-19T10:38:22.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.664+0000 7f2331c12640 1 -- start start 2023-10-19T10:38:22.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.664+0000 7f2331c12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c15c570 0x7f232c0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f2331c12640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f232c0cd920 0x7f232c0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f2331c12640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232c06aa30 con 0x7f232c15c950 2023-10-19T10:38:22.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f2331c12640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f232c06b2d0 con 0x7f232c0cd920 2023-10-19T10:38:22.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f2331c12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0ce090 con 0x7f232c15c570 2023-10-19T10:38:22.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f232c0cd920 0x7f232c0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f232c0cd920 0x7f232c0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42152/0 (socket says 172.21.15.50:42152) 2023-10-19T10:38:22.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 -- 172.21.15.50:0/3732874476 learned_addr learned my addr 172.21.15.50:0/3732874476 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:22.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232b7fe640 1 --2- 172.21.15.50:0/3732874476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c15c570 0x7f232c0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 -- 172.21.15.50:0/3732874476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c15c570 msgr2=0x7f232c0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:22.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 --2- 172.21.15.50:0/3732874476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c15c570 0x7f232c0d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 -- 172.21.15.50:0/3732874476 >> v1:172.21.15.183:6789/0 conn(0x7f232c15c950 legacy=0x7f232c06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.665+0000 7f232bfff640 1 -- 172.21.15.50:0/3732874476 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c06ac80 con 0x7f232c0cd920 2023-10-19T10:38:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.666+0000 7f232bfff640 1 --2- 172.21.15.50:0/3732874476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c0cd920 0x7f232c0d26a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2324009e50 tx=0x7f232402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d0e98f57ea541f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.666+0000 7f232a7fc640 1 -- 172.21.15.50:0/3732874476 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232402ca70 con 0x7f232c0cd920 2023-10-19T10:38:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.666+0000 7f232a7fc640 1 -- 172.21.15.50:0/3732874476 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232402cbd0 con 0x7f232c0cd920 2023-10-19T10:38:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.666+0000 7f232a7fc640 1 -- 172.21.15.50:0/3732874476 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23240343f0 con 0x7f232c0cd920 2023-10-19T10:38:22.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.666+0000 7f2331c12640 1 -- 172.21.15.50:0/3732874476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c0cd920 msgr2=0x7f232c0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.666+0000 7f2331c12640 1 --2- 172.21.15.50:0/3732874476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c0cd920 0x7f232c0d26a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2324009e50 tx=0x7f232402bed0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.667+0000 7f2331c12640 1 -- 172.21.15.50:0/3732874476 shutdown_connections 2023-10-19T10:38:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.667+0000 7f2331c12640 1 --2- 172.21.15.50:0/3732874476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c0cd920 0x7f232c0d26a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.667+0000 7f2331c12640 1 --2- 172.21.15.50:0/3732874476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c15c570 0x7f232c0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.667+0000 7f2331c12640 1 -- 172.21.15.50:0/3732874476 >> 172.21.15.50:0/3732874476 conn(0x7f232c089ff0 msgr2=0x7f232c09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:22.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.667+0000 7f2331c12640 1 -- 172.21.15.50:0/3732874476 shutdown_connections 2023-10-19T10:38:22.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.667+0000 7f2331c12640 1 -- 172.21.15.50:0/3732874476 wait complete. 2023-10-19T10:38:22.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.668+0000 7f2331c12640 1 Processor -- start 2023-10-19T10:38:22.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.668+0000 7f2331c12640 1 -- start start 2023-10-19T10:38:22.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c0cd920 0x7f232c1e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c15c570 0x7f232c1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 0x7f232c0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0b6690 con 0x7f232c15c950 2023-10-19T10:38:22.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0b57e0 con 0x7f232c15c570 2023-10-19T10:38:22.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0b5a80 con 0x7f232c0cd920 2023-10-19T10:38:22.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 0x7f232c0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 0x7f232c0b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42156/0 (socket says 172.21.15.50:42156) 2023-10-19T10:38:22.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 -- 172.21.15.50:0/1851900934 learned_addr learned my addr 172.21.15.50:0/1851900934 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:22.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232b7fe640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c0cd920 0x7f232c1e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232affd640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c15c570 0x7f232c1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 -- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c0cd920 msgr2=0x7f232c1e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c0cd920 0x7f232c1e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 -- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c15c570 msgr2=0x7f232c1597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c15c570 0x7f232c1597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 -- 172.21.15.50:0/1851900934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2324009b30 con 0x7f232c15c950 2023-10-19T10:38:22.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232b7fe640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c0cd920 0x7f232c1e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:22.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f232bfff640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 0x7f232c0b52a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2324036040 tx=0x7f2324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:22.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324009510 con 0x7f232c15c950 2023-10-19T10:38:22.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232403e450 con 0x7f232c15c950 2023-10-19T10:38:22.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c0b5d60 con 0x7f232c15c950 2023-10-19T10:38:22.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232403e800 con 0x7f232c15c950 2023-10-19T10:38:22.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.669+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c0b6320 con 0x7f232c15c950 2023-10-19T10:38:22.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.670+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232c0b6800 con 0x7f232c15c950 2023-10-19T10:38:22.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.671+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f232403e960 con 0x7f232c15c950 2023-10-19T10:38:22.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.671+0000 7f2313fff640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f231806a050 0x7f231806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:22.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.671+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f23240ab0f0 con 0x7f232c15c950 2023-10-19T10:38:22.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.671+0000 7f232b7fe640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f231806a050 0x7f231806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:22.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.672+0000 7f232b7fe640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f231806a050 0x7f231806c510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2320009b80 tx=0x7f2320008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:22.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.676+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2324076950 con 0x7f232c15c950 2023-10-19T10:38:22.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.919+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f232c073990 con 0x7f232c15c950 2023-10-19T10:38:22.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.923+0000 7f2313fff640 1 -- 172.21.15.50:0/1851900934 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2324079f70 con 0x7f232c15c950 2023-10-19T10:38:22.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.924+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f231806a050 msgr2=0x7f231806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.924+0000 7f2331c12640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f231806a050 0x7f231806c510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2320009b80 tx=0x7f2320008040 comp rx=0 tx=0).stop 2023-10-19T10:38:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 msgr2=0x7f232c0b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 0x7f232c0b52a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2324036040 tx=0x7f2324004a40 comp rx=0 tx=0).stop 2023-10-19T10:38:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 shutdown_connections 2023-10-19T10:38:22.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f231806a050 0x7f231806c510 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c15c950 0x7f232c0b52a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c15c570 0x7f232c1597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 --2- 172.21.15.50:0/1851900934 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f232c0cd920 0x7f232c1e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:22.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 >> 172.21.15.50:0/1851900934 conn(0x7f232c089ff0 msgr2=0x7f232c0cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:22.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 shutdown_connections 2023-10-19T10:38:22.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:22.925+0000 7f2331c12640 1 -- 172.21.15.50:0/1851900934 wait complete. 2023-10-19T10:38:22.938 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-19T10:38:23.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.043+0000 7f3fc504d640 1 Processor -- start 2023-10-19T10:38:23.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.043+0000 7f3fc504d640 1 -- start start 2023-10-19T10:38:23.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fc504d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fc00b1f30 0x7f3fc00b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fc504d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3fc00ce080 0x7f3fc00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fc504d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc00caf80 con 0x7f3fc015a370 2023-10-19T10:38:23.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fc504d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc00b2900 con 0x7f3fc00b1f30 2023-10-19T10:38:23.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fc504d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc00d02e0 con 0x7f3fc00ce080 2023-10-19T10:38:23.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fc00b1f30 0x7f3fc00b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fc00b1f30 0x7f3fc00b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42162/0 (socket says 172.21.15.50:42162) 2023-10-19T10:38:23.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 -- 172.21.15.50:0/3830262276 learned_addr learned my addr 172.21.15.50:0/3830262276 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:23.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fbf59e640 1 --2- 172.21.15.50:0/3830262276 >> v2:172.21.15.183:3300/0 conn(0x7f3fc00ce080 0x7f3fc00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 -- 172.21.15.50:0/3830262276 >> v2:172.21.15.183:3300/0 conn(0x7f3fc00ce080 msgr2=0x7f3fc00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:23.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 --2- 172.21.15.50:0/3830262276 >> v2:172.21.15.183:3300/0 conn(0x7f3fc00ce080 0x7f3fc00ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 -- 172.21.15.50:0/3830262276 >> v1:172.21.15.183:6789/0 conn(0x7f3fc015a370 legacy=0x7f3fc006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.044+0000 7f3fb659c640 1 -- 172.21.15.50:0/3830262276 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc015a750 con 0x7f3fc00b1f30 2023-10-19T10:38:23.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.045+0000 7f3fb659c640 1 --2- 172.21.15.50:0/3830262276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc00b1f30 0x7f3fc00b23c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00cce10 tx=0x7f3fa802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fac80b40dbb0ad3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.045+0000 7f3fbe59c640 1 -- 172.21.15.50:0/3830262276 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8030860 con 0x7f3fc00b1f30 2023-10-19T10:38:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.045+0000 7f3fbe59c640 1 -- 172.21.15.50:0/3830262276 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa80309c0 con 0x7f3fc00b1f30 2023-10-19T10:38:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.045+0000 7f3fbe59c640 1 -- 172.21.15.50:0/3830262276 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8030d50 con 0x7f3fc00b1f30 2023-10-19T10:38:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.045+0000 7f3fc504d640 1 -- 172.21.15.50:0/3830262276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc00b1f30 msgr2=0x7f3fc00b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.045+0000 7f3fc504d640 1 --2- 172.21.15.50:0/3830262276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc00b1f30 0x7f3fc00b23c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00cce10 tx=0x7f3fa802b9a0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.046+0000 7f3fc504d640 1 -- 172.21.15.50:0/3830262276 shutdown_connections 2023-10-19T10:38:23.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.046+0000 7f3fc504d640 1 --2- 172.21.15.50:0/3830262276 >> v2:172.21.15.183:3300/0 conn(0x7f3fc00ce080 0x7f3fc00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.046+0000 7f3fc504d640 1 --2- 172.21.15.50:0/3830262276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc00b1f30 0x7f3fc00b23c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.046+0000 7f3fc504d640 1 -- 172.21.15.50:0/3830262276 >> 172.21.15.50:0/3830262276 conn(0x7f3fc00ccf30 msgr2=0x7f3fc00cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:23.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.046+0000 7f3fc504d640 1 -- 172.21.15.50:0/3830262276 shutdown_connections 2023-10-19T10:38:23.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.046+0000 7f3fc504d640 1 -- 172.21.15.50:0/3830262276 wait complete. 2023-10-19T10:38:23.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.047+0000 7f3fc504d640 1 Processor -- start 2023-10-19T10:38:23.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.047+0000 7f3fc504d640 1 -- start start 2023-10-19T10:38:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fc504d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc00b1f30 0x7f3fc015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fc504d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fc00ce080 0x7f3fc00a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fc504d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 0x7f3fc00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fc504d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc00c8b90 con 0x7f3fc015a370 2023-10-19T10:38:23.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fc504d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc00c8820 con 0x7f3fc00b1f30 2023-10-19T10:38:23.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fc504d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015cc90 con 0x7f3fc00ce080 2023-10-19T10:38:23.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 0x7f3fc00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 0x7f3fc00c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42168/0 (socket says 172.21.15.50:42168) 2023-10-19T10:38:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 -- 172.21.15.50:0/4040802558 learned_addr learned my addr 172.21.15.50:0/4040802558 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fb659c640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fc00ce080 0x7f3fc00a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbed9d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc00b1f30 0x7f3fc015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 -- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fc00ce080 msgr2=0x7f3fc00a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fc00ce080 0x7f3fc00a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 -- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc00b1f30 msgr2=0x7f3fc015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc00b1f30 0x7f3fc015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 -- 172.21.15.50:0/4040802558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa80094d0 con 0x7f3fc015a370 2023-10-19T10:38:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fb659c640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fc00ce080 0x7f3fc00a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fbf59e640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 0x7f3fc00c8fb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8007e60 tx=0x7f3fb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.048+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8014420 con 0x7f3fc015a370 2023-10-19T10:38:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.049+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb80145a0 con 0x7f3fc015a370 2023-10-19T10:38:23.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.049+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc015cf70 con 0x7f3fc015a370 2023-10-19T10:38:23.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.049+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8014950 con 0x7f3fc015a370 2023-10-19T10:38:23.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.049+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc015c8d0 con 0x7f3fc015a370 2023-10-19T10:38:23.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.049+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc0073990 con 0x7f3fc015a370 2023-10-19T10:38:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.050+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3fb8014ab0 con 0x7f3fc015a370 2023-10-19T10:38:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.050+0000 7f3fb7fff640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fa006a050 0x7f3fa006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.050+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3fb8088c40 con 0x7f3fc015a370 2023-10-19T10:38:23.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.051+0000 7f3fbed9d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fa006a050 0x7f3fa006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.051+0000 7f3fbed9d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fa006a050 0x7f3fa006c510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0000f60 tx=0x7f3fb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.055+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3fb8058060 con 0x7f3fc015a370 2023-10-19T10:38:23.298 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.296+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3fc00afb20 con 0x7f3fc015a370 2023-10-19T10:38:23.302 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.299+0000 7f3fb7fff640 1 -- 172.21.15.50:0/4040802558 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3fb8057bf0 con 0x7f3fc015a370 2023-10-19T10:38:23.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fa006a050 msgr2=0x7f3fa006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fa006a050 0x7f3fa006c510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0000f60 tx=0x7f3fb0009040 comp rx=0 tx=0).stop 2023-10-19T10:38:23.304 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 msgr2=0x7f3fc00c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 0x7f3fc00c8fb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8007e60 tx=0x7f3fb800e4b0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 shutdown_connections 2023-10-19T10:38:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3fa006a050 0x7f3fa006c510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fc015a370 0x7f3fc00c8fb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fc00ce080 0x7f3fc00a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 --2- 172.21.15.50:0/4040802558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc00b1f30 0x7f3fc015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 >> 172.21.15.50:0/4040802558 conn(0x7f3fc00ccf30 msgr2=0x7f3fc00b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 shutdown_connections 2023-10-19T10:38:23.307 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.301+0000 7f3fc504d640 1 -- 172.21.15.50:0/4040802558 wait complete. 2023-10-19T10:38:23.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-19T10:38:23.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 Processor -- start 2023-10-19T10:38:23.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 -- start start 2023-10-19T10:38:23.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcce40cd7d0 0x7fcce415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce415d2d0 con 0x7fcce4159280 2023-10-19T10:38:23.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce415a820 con 0x7fcce40cd7d0 2023-10-19T10:38:23.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.419+0000 7fccea53e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40ca480 con 0x7fcce4159e90 2023-10-19T10:38:23.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcce40cd7d0 0x7fcce415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcce40cd7d0 0x7fcce415cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42176/0 (socket says 172.21.15.50:42176) 2023-10-19T10:38:23.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce3fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fcce4159280 0x7fcce40c8660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40012/0 (socket says 172.21.15.50:40012) 2023-10-19T10:38:23.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 -- 172.21.15.50:0/2141424034 learned_addr learned my addr 172.21.15.50:0/2141424034 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:23.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce8adb640 1 --2- 172.21.15.50:0/2141424034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 -- 172.21.15.50:0/2141424034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 msgr2=0x7fcce415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:23.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 --2- 172.21.15.50:0/2141424034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 -- 172.21.15.50:0/2141424034 >> v1:172.21.15.50:6789/0 conn(0x7fcce4159280 legacy=0x7fcce40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce37fe640 1 -- 172.21.15.50:0/2141424034 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4159990 con 0x7fcce40cd7d0 2023-10-19T10:38:23.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.420+0000 7fcce2ffd640 1 -- 172.21.15.50:0/2141424034 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854893160 0 0) 0x7fcce415d2d0 con 0x7fcce4159280 2023-10-19T10:38:23.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fcce37fe640 1 --2- 172.21.15.50:0/2141424034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce40cd7d0 0x7fcce415cd90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fccd800bb10 tx=0x7fccd802e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173cf62069922062 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fcce2ffd640 1 -- 172.21.15.50:0/2141424034 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8036440 con 0x7fcce40cd7d0 2023-10-19T10:38:23.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fcce2ffd640 1 -- 172.21.15.50:0/2141424034 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd80365a0 con 0x7fcce40cd7d0 2023-10-19T10:38:23.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fcce2ffd640 1 -- 172.21.15.50:0/2141424034 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8036930 con 0x7fcce40cd7d0 2023-10-19T10:38:23.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fccea53e640 1 -- 172.21.15.50:0/2141424034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce40cd7d0 msgr2=0x7fcce415cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fccea53e640 1 --2- 172.21.15.50:0/2141424034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce40cd7d0 0x7fcce415cd90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fccd800bb10 tx=0x7fccd802e150 comp rx=0 tx=0).stop 2023-10-19T10:38:23.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fccea53e640 1 -- 172.21.15.50:0/2141424034 shutdown_connections 2023-10-19T10:38:23.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fccea53e640 1 --2- 172.21.15.50:0/2141424034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fccea53e640 1 --2- 172.21.15.50:0/2141424034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce40cd7d0 0x7fcce415cd90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.421+0000 7fccea53e640 1 -- 172.21.15.50:0/2141424034 >> 172.21.15.50:0/2141424034 conn(0x7fcce4069e90 msgr2=0x7fcce40afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:23.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.422+0000 7fccea53e640 1 -- 172.21.15.50:0/2141424034 shutdown_connections 2023-10-19T10:38:23.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.422+0000 7fccea53e640 1 -- 172.21.15.50:0/2141424034 wait complete. 2023-10-19T10:38:23.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.423+0000 7fccea53e640 1 Processor -- start 2023-10-19T10:38:23.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.423+0000 7fccea53e640 1 -- start start 2023-10-19T10:38:23.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcce40cd7d0 0x7fcce40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 0x7fcce414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce4152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41519e0 con 0x7fcce4159280 2023-10-19T10:38:23.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40b24c0 con 0x7fcce40cd7d0 2023-10-19T10:38:23.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce414fa50 con 0x7fcce4159e90 2023-10-19T10:38:23.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 0x7fcce414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 0x7fcce414f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42190/0 (socket says 172.21.15.50:42190) 2023-10-19T10:38:23.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 -- 172.21.15.50:0/266076284 learned_addr learned my addr 172.21.15.50:0/266076284 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:23.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce8adb640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce4152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce3fff640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcce40cd7d0 0x7fcce40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 -- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 msgr2=0x7fcce4152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce4152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 -- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcce40cd7d0 msgr2=0x7fcce40b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcce40cd7d0 0x7fcce40b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 -- 172.21.15.50:0/266076284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd800b770 con 0x7fcce4159280 2023-10-19T10:38:23.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce3fff640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcce40cd7d0 0x7fcce40b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:23.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce37fe640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 0x7fcce414f510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fccd802e190 tx=0x7fccd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8036440 con 0x7fcce4159280 2023-10-19T10:38:23.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8034ba0 con 0x7fcce4159280 2023-10-19T10:38:23.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce40b0c40 con 0x7fcce4159280 2023-10-19T10:38:23.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8041440 con 0x7fcce4159280 2023-10-19T10:38:23.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.424+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce40b11c0 con 0x7fcce4159280 2023-10-19T10:38:23.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.425+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce40c19b0 con 0x7fcce4159280 2023-10-19T10:38:23.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.426+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fccd803d040 con 0x7fcce4159280 2023-10-19T10:38:23.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.426+0000 7fcce0ff9640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fccd006a050 0x7fccd006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.426+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fccd80ad160 con 0x7fcce4159280 2023-10-19T10:38:23.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.427+0000 7fcce3fff640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fccd006a050 0x7fccd006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.427+0000 7fcce3fff640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fccd006a050 0x7fccd006c510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccd4000f60 tx=0x7fccd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.431+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fccd8078a60 con 0x7fcce4159280 2023-10-19T10:38:23.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.680+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fcce4073990 con 0x7fcce4159280 2023-10-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.683+0000 7fcce0ff9640 1 -- 172.21.15.50:0/266076284 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fccd807c080 con 0x7fcce4159280 2023-10-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fccd006a050 msgr2=0x7fccd006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fccd006a050 0x7fccd006c510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccd4000f60 tx=0x7fccd4009040 comp rx=0 tx=0).stop 2023-10-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 msgr2=0x7fcce414f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 0x7fcce414f510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fccd802e190 tx=0x7fccd80046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 shutdown_connections 2023-10-19T10:38:23.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fccd006a050 0x7fccd006c510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcce4159e90 0x7fcce4152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce4159280 0x7fcce414f510 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 --2- 172.21.15.50:0/266076284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcce40cd7d0 0x7fcce40b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.685+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 >> 172.21.15.50:0/266076284 conn(0x7fcce4069e90 msgr2=0x7fcce40cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:23.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.686+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 shutdown_connections 2023-10-19T10:38:23.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.686+0000 7fccea53e640 1 -- 172.21.15.50:0/266076284 wait complete. 2023-10-19T10:38:23.698 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-19T10:38:23.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.803+0000 7f665a41c640 1 Processor -- start 2023-10-19T10:38:23.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.803+0000 7f665a41c640 1 -- start start 2023-10-19T10:38:23.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f665a41c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66540d0e30 0x7f665415c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f665a41c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f66540d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f665a41c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665406aa30 con 0x7f665415c570 2023-10-19T10:38:23.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f665a41c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665415ce90 con 0x7f66540d0e30 2023-10-19T10:38:23.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f665a41c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665406b2d0 con 0x7f66540cd920 2023-10-19T10:38:23.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66540d0e30 0x7f665415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66540d0e30 0x7f665415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42200/0 (socket says 172.21.15.50:42200) 2023-10-19T10:38:23.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 -- 172.21.15.50:0/4207645829 learned_addr learned my addr 172.21.15.50:0/4207645829 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:23.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66589b9640 1 --2- 172.21.15.50:0/4207645829 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f66540d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 -- 172.21.15.50:0/4207645829 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 msgr2=0x7f66540d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:23.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 --2- 172.21.15.50:0/4207645829 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f66540d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 -- 172.21.15.50:0/4207645829 >> v1:172.21.15.183:6789/0 conn(0x7f665415c570 legacy=0x7f665406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:23.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.804+0000 7f66537fe640 1 -- 172.21.15.50:0/4207645829 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540d1270 con 0x7f66540d0e30 2023-10-19T10:38:23.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.805+0000 7f66537fe640 1 --2- 172.21.15.50:0/4207645829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f665415c950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6644009870 tx=0x7f664402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a9bf98a01fd73c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.805+0000 7f6652ffd640 1 -- 172.21.15.50:0/4207645829 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440317c0 con 0x7f66540d0e30 2023-10-19T10:38:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.805+0000 7f6652ffd640 1 -- 172.21.15.50:0/4207645829 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644031920 con 0x7f66540d0e30 2023-10-19T10:38:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.805+0000 7f6652ffd640 1 -- 172.21.15.50:0/4207645829 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644031cb0 con 0x7f66540d0e30 2023-10-19T10:38:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 -- 172.21.15.50:0/4207645829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 msgr2=0x7f665415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 --2- 172.21.15.50:0/4207645829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f665415c950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6644009870 tx=0x7f664402b860 comp rx=0 tx=0).stop 2023-10-19T10:38:23.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 -- 172.21.15.50:0/4207645829 shutdown_connections 2023-10-19T10:38:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 --2- 172.21.15.50:0/4207645829 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f66540d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 --2- 172.21.15.50:0/4207645829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f665415c950 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 -- 172.21.15.50:0/4207645829 >> 172.21.15.50:0/4207645829 conn(0x7f6654089ff0 msgr2=0x7f665409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 -- 172.21.15.50:0/4207645829 shutdown_connections 2023-10-19T10:38:23.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.806+0000 7f665a41c640 1 -- 172.21.15.50:0/4207645829 wait complete. 2023-10-19T10:38:23.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.807+0000 7f665a41c640 1 Processor -- start 2023-10-19T10:38:23.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.807+0000 7f665a41c640 1 -- start start 2023-10-19T10:38:23.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f665a41c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f665414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f665a41c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f66540b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f665a41c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415c570 0x7f66541e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f665a41c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6654158ea0 con 0x7f66540d0e30 2023-10-19T10:38:23.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f665a41c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66540b52a0 con 0x7f665415c570 2023-10-19T10:38:23.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f665a41c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66540b5a80 con 0x7f66540cd920 2023-10-19T10:38:23.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f66540b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f66540b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42212/0 (socket says 172.21.15.50:42212) 2023-10-19T10:38:23.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 -- 172.21.15.50:0/3867383508 learned_addr learned my addr 172.21.15.50:0/3867383508 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:23.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66589b9640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415c570 0x7f66541e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f6653fff640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f665414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 -- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 msgr2=0x7f665414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f665414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 -- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415c570 msgr2=0x7f66541e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:23.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415c570 0x7f66541e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:23.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.808+0000 7f66537fe640 1 -- 172.21.15.50:0/3867383508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440094d0 con 0x7f66540d0e30 2023-10-19T10:38:23.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.809+0000 7f66537fe640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f66540b7a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6644000c00 tx=0x7f66440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.809+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440329a0 con 0x7f66540d0e30 2023-10-19T10:38:23.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.809+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644032b20 con 0x7f66540d0e30 2023-10-19T10:38:23.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.809+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440453f0 con 0x7f66540d0e30 2023-10-19T10:38:23.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.809+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6654159140 con 0x7f66540d0e30 2023-10-19T10:38:23.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.809+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66540b5560 con 0x7f66540d0e30 2023-10-19T10:38:23.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.810+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66540b1580 con 0x7f66540d0e30 2023-10-19T10:38:23.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.810+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f664404d3f0 con 0x7f66540d0e30 2023-10-19T10:38:23.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.810+0000 7f6650ff9640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f664006a050 0x7f664006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:23.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.811+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f66440aa370 con 0x7f66540d0e30 2023-10-19T10:38:23.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.811+0000 7f6653fff640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f664006a050 0x7f664006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:23.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.811+0000 7f6653fff640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f664006a050 0x7f664006c510 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6648004e50 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:23.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:23.815+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6644075d90 con 0x7f66540d0e30 2023-10-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.058+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f66540d3860 con 0x7f66540d0e30 2023-10-19T10:38:24.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.062+0000 7f6650ff9640 1 -- 172.21.15.50:0/3867383508 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f66440793b0 con 0x7f66540d0e30 2023-10-19T10:38:24.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f664006a050 msgr2=0x7f664006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f664006a050 0x7f664006c510 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6648004e50 tx=0x7f6648009040 comp rx=0 tx=0).stop 2023-10-19T10:38:24.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 msgr2=0x7f66540b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f66540b7a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6644000c00 tx=0x7f66440046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 shutdown_connections 2023-10-19T10:38:24.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f664006a050 0x7f664006c510 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f665415c570 0x7f66541e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66540d0e30 0x7f66540b7a20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 --2- 172.21.15.50:0/3867383508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66540cd920 0x7f665414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 >> 172.21.15.50:0/3867383508 conn(0x7f6654089ff0 msgr2=0x7f665415bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:24.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 shutdown_connections 2023-10-19T10:38:24.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.064+0000 7f665a41c640 1 -- 172.21.15.50:0/3867383508 wait complete. 2023-10-19T10:38:24.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-19T10:38:24.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.181+0000 7fd765ee9640 1 Processor -- start 2023-10-19T10:38:24.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.181+0000 7fd765ee9640 1 -- start start 2023-10-19T10:38:24.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd765ee9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600d0e30 0x7fd76015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd765ee9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd7600cd920 0x7fd7600d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd765ee9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76006aa30 con 0x7fd76015c570 2023-10-19T10:38:24.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd765ee9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76015ce90 con 0x7fd7600cd920 2023-10-19T10:38:24.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd765ee9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76006b2d0 con 0x7fd7600d0e30 2023-10-19T10:38:24.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd7600cd920 0x7fd7600d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd7600cd920 0x7fd7600d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42224/0 (socket says 172.21.15.50:42224) 2023-10-19T10:38:24.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 -- 172.21.15.50:0/3975151239 learned_addr learned my addr 172.21.15.50:0/3975151239 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:24.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75effd640 1 --2- 172.21.15.50:0/3975151239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600d0e30 0x7fd76015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 -- 172.21.15.50:0/3975151239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600d0e30 msgr2=0x7fd76015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:24.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 --2- 172.21.15.50:0/3975151239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600d0e30 0x7fd76015c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 -- 172.21.15.50:0/3975151239 >> v1:172.21.15.183:6789/0 conn(0x7fd76015c570 legacy=0x7fd76006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.182+0000 7fd75ffff640 1 -- 172.21.15.50:0/3975151239 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600d1270 con 0x7fd7600cd920 2023-10-19T10:38:24.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.183+0000 7fd75ffff640 1 --2- 172.21.15.50:0/3975151239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600cd920 0x7fd7600d26a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd758009e50 tx=0x7fd75802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162c7528a8ce3140 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.183+0000 7fd75e7fc640 1 -- 172.21.15.50:0/3975151239 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75802ca70 con 0x7fd7600cd920 2023-10-19T10:38:24.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.183+0000 7fd75e7fc640 1 -- 172.21.15.50:0/3975151239 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75802cbd0 con 0x7fd7600cd920 2023-10-19T10:38:24.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.183+0000 7fd75e7fc640 1 -- 172.21.15.50:0/3975151239 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580343f0 con 0x7fd7600cd920 2023-10-19T10:38:24.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 -- 172.21.15.50:0/3975151239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600cd920 msgr2=0x7fd7600d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3975151239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600cd920 0x7fd7600d26a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd758009e50 tx=0x7fd75802bed0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 -- 172.21.15.50:0/3975151239 shutdown_connections 2023-10-19T10:38:24.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3975151239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600cd920 0x7fd7600d26a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3975151239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600d0e30 0x7fd76015c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 -- 172.21.15.50:0/3975151239 >> 172.21.15.50:0/3975151239 conn(0x7fd760089ff0 msgr2=0x7fd76009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:24.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 -- 172.21.15.50:0/3975151239 shutdown_connections 2023-10-19T10:38:24.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.184+0000 7fd765ee9640 1 -- 172.21.15.50:0/3975151239 wait complete. 2023-10-19T10:38:24.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.185+0000 7fd765ee9640 1 Processor -- start 2023-10-19T10:38:24.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.185+0000 7fd765ee9640 1 -- start start 2023-10-19T10:38:24.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd765ee9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600cd920 0x7fd76015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd765ee9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 0x7fd7601f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd765ee9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76015c570 0x7fd7600cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd765ee9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76015a830 con 0x7fd7600d0e30 2023-10-19T10:38:24.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd765ee9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600b6510 con 0x7fd76015c570 2023-10-19T10:38:24.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd765ee9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7601f20e0 con 0x7fd7600cd920 2023-10-19T10:38:24.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 0x7fd7601f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 0x7fd7601f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42232/0 (socket says 172.21.15.50:42232) 2023-10-19T10:38:24.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 -- 172.21.15.50:0/3642474247 learned_addr learned my addr 172.21.15.50:0/3642474247 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:24.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75ffff640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76015c570 0x7fd7600cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75f7fe640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600cd920 0x7fd76015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 -- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600cd920 msgr2=0x7fd76015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600cd920 0x7fd76015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 -- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76015c570 msgr2=0x7fd7600cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76015c570 0x7fd7600cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 -- 172.21.15.50:0/3642474247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758009b30 con 0x7fd7600d0e30 2023-10-19T10:38:24.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75ffff640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76015c570 0x7fd7600cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:24.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75f7fe640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600cd920 0x7fd76015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:24.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.186+0000 7fd75effd640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 0x7fd7601f1820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd75400e870 tx=0x7fd75400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.187+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7540094f0 con 0x7fd7600d0e30 2023-10-19T10:38:24.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.187+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75400c940 con 0x7fd7600d0e30 2023-10-19T10:38:24.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.187+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7601f23c0 con 0x7fd7600d0e30 2023-10-19T10:38:24.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.187+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400ccf0 con 0x7fd7600d0e30 2023-10-19T10:38:24.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.187+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7600b67b0 con 0x7fd7600d0e30 2023-10-19T10:38:24.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.188+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd760073990 con 0x7fd7600d0e30 2023-10-19T10:38:24.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.188+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd754013470 con 0x7fd7600d0e30 2023-10-19T10:38:24.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.189+0000 7fd747fff640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd74c06a050 0x7fd74c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.189+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd7540884d0 con 0x7fd7600d0e30 2023-10-19T10:38:24.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.189+0000 7fd75f7fe640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd74c06a050 0x7fd74c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.189+0000 7fd75f7fe640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd74c06a050 0x7fd74c06c510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd750000fb0 tx=0x7fd750009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.193+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd754053dd0 con 0x7fd7600d0e30 2023-10-19T10:38:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.433+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd7600d3480 con 0x7fd7600d0e30 2023-10-19T10:38:24.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.436+0000 7fd747fff640 1 -- 172.21.15.50:0/3642474247 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd7540573f0 con 0x7fd7600d0e30 2023-10-19T10:38:24.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd74c06a050 msgr2=0x7fd74c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd74c06a050 0x7fd74c06c510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd750000fb0 tx=0x7fd750009040 comp rx=0 tx=0).stop 2023-10-19T10:38:24.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 msgr2=0x7fd7601f1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 0x7fd7601f1820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd75400e870 tx=0x7fd75400ed30 comp rx=0 tx=0).stop 2023-10-19T10:38:24.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 shutdown_connections 2023-10-19T10:38:24.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd74c06a050 0x7fd74c06c510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd76015c570 0x7fd7600cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7600d0e30 0x7fd7601f1820 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 --2- 172.21.15.50:0/3642474247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7600cd920 0x7fd76015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 >> 172.21.15.50:0/3642474247 conn(0x7fd760089ff0 msgr2=0x7fd7600cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:24.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 shutdown_connections 2023-10-19T10:38:24.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.438+0000 7fd765ee9640 1 -- 172.21.15.50:0/3642474247 wait complete. 2023-10-19T10:38:24.452 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-19T10:38:24.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.557+0000 7fc845f7f640 1 Processor -- start 2023-10-19T10:38:24.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.557+0000 7fc845f7f640 1 -- start start 2023-10-19T10:38:24.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc845f7f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc84015b780 0x7fc84015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc845f7f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc8400c27d0 0x7fc840069800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc845f7f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84015f050 con 0x7fc8400ae750 2023-10-19T10:38:24.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc845f7f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8400cb990 con 0x7fc84015b780 2023-10-19T10:38:24.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc845f7f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84015e0c0 con 0x7fc8400c27d0 2023-10-19T10:38:24.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc83f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc84015b780 0x7fc84015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc83f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc84015b780 0x7fc84015eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42246/0 (socket says 172.21.15.50:42246) 2023-10-19T10:38:24.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc83ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc8400ae750 0x7fc840070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40028/0 (socket says 172.21.15.50:40028) 2023-10-19T10:38:24.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.558+0000 7fc83f7fe640 1 -- 172.21.15.50:0/1980470092 learned_addr learned my addr 172.21.15.50:0/1980470092 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:24.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.559+0000 7fc83effd640 1 --2- 172.21.15.50:0/1980470092 >> v2:172.21.15.183:3300/0 conn(0x7fc8400c27d0 0x7fc840069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.559+0000 7fc83f7fe640 1 -- 172.21.15.50:0/1980470092 >> v2:172.21.15.183:3300/0 conn(0x7fc8400c27d0 msgr2=0x7fc840069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:24.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.559+0000 7fc83f7fe640 1 --2- 172.21.15.50:0/1980470092 >> v2:172.21.15.183:3300/0 conn(0x7fc8400c27d0 0x7fc840069800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.559+0000 7fc83f7fe640 1 -- 172.21.15.50:0/1980470092 >> v1:172.21.15.50:6789/0 conn(0x7fc8400ae750 legacy=0x7fc840070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.559+0000 7fc83e7fc640 1 -- 172.21.15.50:0/1980470092 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666591312 0 0) 0x7fc838001490 con 0x7fc8400ae750 2023-10-19T10:38:24.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.559+0000 7fc83f7fe640 1 -- 172.21.15.50:0/1980470092 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400cc850 con 0x7fc84015b780 2023-10-19T10:38:24.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc83f7fe640 1 --2- 172.21.15.50:0/1980470092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc84015eb10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8400738f0 tx=0x7fc83002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc2fac980b24cfc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc83e7fc640 1 -- 172.21.15.50:0/1980470092 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300317e0 con 0x7fc84015b780 2023-10-19T10:38:24.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc83e7fc640 1 -- 172.21.15.50:0/1980470092 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830031940 con 0x7fc84015b780 2023-10-19T10:38:24.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc83e7fc640 1 -- 172.21.15.50:0/1980470092 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830031cd0 con 0x7fc84015b780 2023-10-19T10:38:24.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 -- 172.21.15.50:0/1980470092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 msgr2=0x7fc84015eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 --2- 172.21.15.50:0/1980470092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc84015eb10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8400738f0 tx=0x7fc83002b880 comp rx=0 tx=0).stop 2023-10-19T10:38:24.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 -- 172.21.15.50:0/1980470092 shutdown_connections 2023-10-19T10:38:24.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 --2- 172.21.15.50:0/1980470092 >> v2:172.21.15.183:3300/0 conn(0x7fc8400c27d0 0x7fc840069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 --2- 172.21.15.50:0/1980470092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc84015eb10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 -- 172.21.15.50:0/1980470092 >> 172.21.15.50:0/1980470092 conn(0x7fc8400cbbc0 msgr2=0x7fc8400cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:24.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 -- 172.21.15.50:0/1980470092 shutdown_connections 2023-10-19T10:38:24.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.560+0000 7fc845f7f640 1 -- 172.21.15.50:0/1980470092 wait complete. 2023-10-19T10:38:24.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.561+0000 7fc845f7f640 1 Processor -- start 2023-10-19T10:38:24.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 -- start start 2023-10-19T10:38:24.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8400ae750 0x7fc840148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8400c27d0 0x7fc8401516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc840151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc840148fe0 con 0x7fc84015b780 2023-10-19T10:38:24.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401584d0 con 0x7fc8400c27d0 2023-10-19T10:38:24.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc845f7f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc840157900 con 0x7fc8400ae750 2023-10-19T10:38:24.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc840151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc840151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42262/0 (socket says 172.21.15.50:42262) 2023-10-19T10:38:24.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.562+0000 7fc83ffff640 1 -- 172.21.15.50:0/2973862400 learned_addr learned my addr 172.21.15.50:0/2973862400 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:24.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83effd640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8400c27d0 0x7fc8401516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83f7fe640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8400ae750 0x7fc840148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83ffff640 1 -- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8400ae750 msgr2=0x7fc840148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83ffff640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8400ae750 0x7fc840148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83ffff640 1 -- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8400c27d0 msgr2=0x7fc8401516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83ffff640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8400c27d0 0x7fc8401516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83ffff640 1 -- 172.21.15.50:0/2973862400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300094d0 con 0x7fc84015b780 2023-10-19T10:38:24.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83f7fe640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8400ae750 0x7fc840148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:24.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc83ffff640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc840151bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c730 tx=0x7fc83800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83800ce40 con 0x7fc84015b780 2023-10-19T10:38:24.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83800a640 con 0x7fc84015b780 2023-10-19T10:38:24.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc840157ba0 con 0x7fc84015b780 2023-10-19T10:38:24.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83800aa30 con 0x7fc84015b780 2023-10-19T10:38:24.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.563+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc840158160 con 0x7fc84015b780 2023-10-19T10:38:24.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.564+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc840071c70 con 0x7fc84015b780 2023-10-19T10:38:24.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.565+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc83800ab90 con 0x7fc84015b780 2023-10-19T10:38:24.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.565+0000 7fc827fff640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc82c06a050 0x7fc82c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.565+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc8380878b0 con 0x7fc84015b780 2023-10-19T10:38:24.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.565+0000 7fc83f7fe640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc82c06a050 0x7fc82c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.566+0000 7fc83f7fe640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc82c06a050 0x7fc82c06c510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc830036000 tx=0x7fc83003d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.569+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc838053240 con 0x7fc84015b780 2023-10-19T10:38:24.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.815+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc84015f050 con 0x7fc84015b780 2023-10-19T10:38:24.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.819+0000 7fc827fff640 1 -- 172.21.15.50:0/2973862400 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc84015f050 con 0x7fc84015b780 2023-10-19T10:38:24.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc82c06a050 msgr2=0x7fc82c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc82c06a050 0x7fc82c06c510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc830036000 tx=0x7fc83003d040 comp rx=0 tx=0).stop 2023-10-19T10:38:24.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 msgr2=0x7fc840151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc840151bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c730 tx=0x7fc83800cbf0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 shutdown_connections 2023-10-19T10:38:24.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc82c06a050 0x7fc82c06c510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.824 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc84015b780 0x7fc840151bf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8400c27d0 0x7fc8401516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 --2- 172.21.15.50:0/2973862400 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8400ae750 0x7fc840148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 >> 172.21.15.50:0/2973862400 conn(0x7fc8400cbbc0 msgr2=0x7fc8400b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:24.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 shutdown_connections 2023-10-19T10:38:24.825 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.821+0000 7fc845f7f640 1 -- 172.21.15.50:0/2973862400 wait complete. 2023-10-19T10:38:24.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:24.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.937+0000 7f543df27640 1 Processor -- start 2023-10-19T10:38:24.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.937+0000 7f543df27640 1 -- start start 2023-10-19T10:38:24.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.938+0000 7f543df27640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 0x7f543815df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.938+0000 7f543df27640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54380cb520 con 0x7f543806f6e0 2023-10-19T10:38:24.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.938+0000 7f543df27640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543815a790 con 0x7f543815a3b0 2023-10-19T10:38:24.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.938+0000 7f543df27640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543815eea0 con 0x7f5438071690 2023-10-19T10:38:24.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.938+0000 7f5437fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f543806f6e0 0x7f54380c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40036/0 (socket says 172.21.15.50:40036) 2023-10-19T10:38:24.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.938+0000 7f5437fff640 1 -- 172.21.15.50:0/2738231185 learned_addr learned my addr 172.21.15.50:0/2738231185 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:24.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5436ffd640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 0x7f543815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237490468 0 0) 0x7f54380cb520 con 0x7f543806f6e0 2023-10-19T10:38:24.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5420003520 con 0x7f543806f6e0 2023-10-19T10:38:24.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2740472014 0 0) 0x7f5420003520 con 0x7f543806f6e0 2023-10-19T10:38:24.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 msgr2=0x7f543815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:24.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 0x7f543815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 >> v1:172.21.15.183:6789/0 conn(0x7f543815a3b0 legacy=0x7f54380b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543815e600 con 0x7f543806f6e0 2023-10-19T10:38:24.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f54300019f0 con 0x7f543806f6e0 2023-10-19T10:38:24.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 >> v1:172.21.15.50:6789/0 conn(0x7f543806f6e0 legacy=0x7f54380c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54200039a0 0x7f5420003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 0x7f54200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420004d00 0x7f5420005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543815eea0 con 0x7f5420004d00 2023-10-19T10:38:24.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542000ba30 con 0x7f54200039a0 2023-10-19T10:38:24.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542000bbf0 con 0x7f5438071690 2023-10-19T10:38:24.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5437fff640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420004d00 0x7f5420005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5436ffd640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 0x7f54200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.948 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f54377fe640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54200039a0 0x7f5420003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5437fff640 1 -- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 msgr2=0x7f54200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:24.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5437fff640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5438071690 0x7f54200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5437fff640 1 -- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54200039a0 msgr2=0x7f5420003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:24.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.939+0000 7f5437fff640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54200039a0 0x7f5420003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.940+0000 7f5437fff640 1 -- 172.21.15.50:0/2738231185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420003520 con 0x7f5420004d00 2023-10-19T10:38:24.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.940+0000 7f54377fe640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54200039a0 0x7f5420003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:24.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.940+0000 7f5437fff640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420004d00 0x7f5420005180 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f543000a800 tx=0x7f54200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9583c15ba56e90 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.940+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300072d0 con 0x7f5420004d00 2023-10-19T10:38:24.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.940+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543002f070 con 0x7f5420004d00 2023-10-19T10:38:24.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.940+0000 7f54367fc640 1 -- 172.21.15.50:0/2738231185 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300076b0 con 0x7f5420004d00 2023-10-19T10:38:24.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f543df27640 1 -- 172.21.15.50:0/2738231185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420004d00 msgr2=0x7f5420005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f543df27640 1 --2- 172.21.15.50:0/2738231185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420004d00 0x7f5420005180 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f543000a800 tx=0x7f54200057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f5437fff640 1 -- 172.21.15.50:0/2738231185 reap_dead start 2023-10-19T10:38:24.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f543df27640 1 -- 172.21.15.50:0/2738231185 shutdown_connections 2023-10-19T10:38:24.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f543df27640 1 -- 172.21.15.50:0/2738231185 >> 172.21.15.50:0/2738231185 conn(0x7f5438069e90 msgr2=0x7f54380afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:24.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f543df27640 1 -- 172.21.15.50:0/2738231185 shutdown_connections 2023-10-19T10:38:24.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.941+0000 7f543df27640 1 -- 172.21.15.50:0/2738231185 wait complete. 2023-10-19T10:38:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.942+0000 7f543df27640 1 Processor -- start 2023-10-19T10:38:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.942+0000 7f543df27640 1 -- start start 2023-10-19T10:38:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f543df27640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f543814e490 0x7f543814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f543df27640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 0x7f54380c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f543df27640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54380af600 0x7f54380afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f543df27640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380c39e0 con 0x7f54380c6e60 2023-10-19T10:38:24.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f543df27640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380affc0 con 0x7f54380af600 2023-10-19T10:38:24.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f543df27640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380c2d60 con 0x7f543814e490 2023-10-19T10:38:24.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f5436ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 0x7f54380c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f5436ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 0x7f54380c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42270/0 (socket says 172.21.15.50:42270) 2023-10-19T10:38:24.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f5436ffd640 1 -- 172.21.15.50:0/1115825349 learned_addr learned my addr 172.21.15.50:0/1115825349 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:24.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f5437fff640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54380af600 0x7f54380afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.943+0000 7f54377fe640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f543814e490 0x7f543814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5436ffd640 1 -- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f543814e490 msgr2=0x7f543814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5436ffd640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f543814e490 0x7f543814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5436ffd640 1 -- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54380af600 msgr2=0x7f54380afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:24.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5436ffd640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54380af600 0x7f54380afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:24.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5436ffd640 1 -- 172.21.15.50:0/1115825349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380c3040 con 0x7f54380c6e60 2023-10-19T10:38:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5437fff640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54380af600 0x7f54380afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f54377fe640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f543814e490 0x7f543814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f5436ffd640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 0x7f54380c2520 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f542c0119a0 tx=0x7f542c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00faf0 con 0x7f54380c6e60 2023-10-19T10:38:24.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c00fc50 con 0x7f54380c6e60 2023-10-19T10:38:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54380c3320 con 0x7f54380c6e60 2023-10-19T10:38:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c014440 con 0x7f54380c6e60 2023-10-19T10:38:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.944+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54380c2a60 con 0x7f54380c6e60 2023-10-19T10:38:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.945+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54380c3b20 con 0x7f54380c6e60 2023-10-19T10:38:24.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.945+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f542c0145a0 con 0x7f54380c6e60 2023-10-19T10:38:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.946+0000 7f541ffff640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f542406a080 0x7f542406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.946+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f542c088b40 con 0x7f54380c6e60 2023-10-19T10:38:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.946+0000 7f54377fe640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f542406a080 0x7f542406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.947+0000 7f54377fe640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f542406a080 0x7f542406c540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f54381525a0 tx=0x7f5428009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:24.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:24.950+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f54380c3b20 con 0x7f54380c6e60 2023-10-19T10:38:25.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.126+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54380c3b20 con 0x7f54380c6e60 2023-10-19T10:38:25.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.127+0000 7f541ffff640 1 -- 172.21.15.50:0/1115825349 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+14808 (secure 0 0 0) 0x7f54380c3b20 con 0x7f54380c6e60 2023-10-19T10:38:25.130 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:25.130 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":38,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:38:22.120308+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:25.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f542406a080 msgr2=0x7f542406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f542406a080 0x7f542406c540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f54381525a0 tx=0x7f5428009040 comp rx=0 tx=0).stop 2023-10-19T10:38:25.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 msgr2=0x7f54380c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 0x7f54380c2520 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f542c0119a0 tx=0x7f542c011e60 comp rx=0 tx=0).stop 2023-10-19T10:38:25.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 shutdown_connections 2023-10-19T10:38:25.132 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f542406a080 0x7f542406c540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54380af600 0x7f54380afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54380c6e60 0x7f54380c2520 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 --2- 172.21.15.50:0/1115825349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f543814e490 0x7f543814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.133 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 >> 172.21.15.50:0/1115825349 conn(0x7f5438069e90 msgr2=0x7f54380b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:25.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 shutdown_connections 2023-10-19T10:38:25.134 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.129+0000 7f543df27640 1 -- 172.21.15.50:0/1115825349 wait complete. 2023-10-19T10:38:25.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:38:25.143 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:38:25.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.249+0000 7fd14b3d0640 1 Processor -- start 2023-10-19T10:38:25.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.249+0000 7fd14b3d0640 1 -- start start 2023-10-19T10:38:25.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14b3d0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd1440b38f0 0x7fd1440d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14b3d0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1440af250 con 0x7fd1440b3090 2023-10-19T10:38:25.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14b3d0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1440af510 con 0x7fd144161210 2023-10-19T10:38:25.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14b3d0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1440b0190 con 0x7fd1440b38f0 2023-10-19T10:38:25.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14916c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd1440b38f0 0x7fd1440d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14896b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd1440b3090 0x7fd144160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40050/0 (socket says 172.21.15.50:40050) 2023-10-19T10:38:25.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14896b640 1 -- 172.21.15.50:0/3699201706 learned_addr learned my addr 172.21.15.50:0/3699201706 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:25.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd143fff640 1 -- 172.21.15.50:0/3699201706 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326584765 0 0) 0x7fd1440af250 con 0x7fd1440b3090 2023-10-19T10:38:25.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd143fff640 1 -- 172.21.15.50:0/3699201706 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12c003520 con 0x7fd1440b3090 2023-10-19T10:38:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14916c640 1 -- 172.21.15.50:0/3699201706 >> v1:172.21.15.183:6789/0 conn(0x7fd144161210 legacy=0x7fd1440b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14916c640 1 -- 172.21.15.50:0/3699201706 >> v1:172.21.15.50:6789/0 conn(0x7fd1440b3090 legacy=0x7fd144160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.250+0000 7fd14916c640 1 -- 172.21.15.50:0/3699201706 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440af7f0 con 0x7fd1440b38f0 2023-10-19T10:38:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.251+0000 7fd14916c640 1 --2- 172.21.15.50:0/3699201706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1440b38f0 0x7fd1440d30e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd134009790 tx=0x7fd13402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf8e551e74ea378 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.251+0000 7fd143fff640 1 -- 172.21.15.50:0/3699201706 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134031830 con 0x7fd1440b38f0 2023-10-19T10:38:25.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.251+0000 7fd143fff640 1 -- 172.21.15.50:0/3699201706 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd134031990 con 0x7fd1440b38f0 2023-10-19T10:38:25.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.251+0000 7fd143fff640 1 -- 172.21.15.50:0/3699201706 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134031d20 con 0x7fd1440b38f0 2023-10-19T10:38:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 -- 172.21.15.50:0/3699201706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1440b38f0 msgr2=0x7fd1440d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/3699201706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1440b38f0 0x7fd1440d30e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd134009790 tx=0x7fd13402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 -- 172.21.15.50:0/3699201706 shutdown_connections 2023-10-19T10:38:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/3699201706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1440b38f0 0x7fd1440d30e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 -- 172.21.15.50:0/3699201706 >> 172.21.15.50:0/3699201706 conn(0x7fd14409b960 msgr2=0x7fd1440d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 -- 172.21.15.50:0/3699201706 shutdown_connections 2023-10-19T10:38:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.252+0000 7fd14b3d0640 1 -- 172.21.15.50:0/3699201706 wait complete. 2023-10-19T10:38:25.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.253+0000 7fd14b3d0640 1 Processor -- start 2023-10-19T10:38:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.253+0000 7fd14b3d0640 1 -- start start 2023-10-19T10:38:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14b3d0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1440b3090 0x7fd14414d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14b3d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1440b38f0 0x7fd14414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14b3d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 0x7fd14414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14b3d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414c8d0 con 0x7fd144161210 2023-10-19T10:38:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14b3d0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414cbb0 con 0x7fd1440b38f0 2023-10-19T10:38:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14b3d0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414ce90 con 0x7fd1440b3090 2023-10-19T10:38:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 0x7fd14414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 0x7fd14414b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42292/0 (socket says 172.21.15.50:42292) 2023-10-19T10:38:25.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 -- 172.21.15.50:0/2876339560 learned_addr learned my addr 172.21.15.50:0/2876339560 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:25.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14916c640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1440b3090 0x7fd14414d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14896b640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1440b38f0 0x7fd14414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 -- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1440b3090 msgr2=0x7fd14414d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1440b3090 0x7fd14414d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 -- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1440b38f0 msgr2=0x7fd14414bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1440b38f0 0x7fd14414bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14996d640 1 -- 172.21.15.50:0/2876339560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340093a0 con 0x7fd144161210 2023-10-19T10:38:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.254+0000 7fd14896b640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1440b38f0 0x7fd14414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.255+0000 7fd14996d640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 0x7fd14414b480 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b9e0 tx=0x7fd13c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.255+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c003b70 con 0x7fd144161210 2023-10-19T10:38:25.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.255+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c003cd0 con 0x7fd144161210 2023-10-19T10:38:25.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.255+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14414d170 con 0x7fd144161210 2023-10-19T10:38:25.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.255+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00cb60 con 0x7fd144161210 2023-10-19T10:38:25.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.255+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14414c510 con 0x7fd144161210 2023-10-19T10:38:25.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.256+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14414d690 con 0x7fd144161210 2023-10-19T10:38:25.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.256+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd13c00ccc0 con 0x7fd144161210 2023-10-19T10:38:25.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.256+0000 7fd141ffb640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd13006a050 0x7fd13006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.257+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd13c088180 con 0x7fd144161210 2023-10-19T10:38:25.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.257+0000 7fd14916c640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd13006a050 0x7fd13006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.267 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.257+0000 7fd14916c640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd13006a050 0x7fd13006c510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd134036000 tx=0x7fd134008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.268 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.261+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd13c053a80 con 0x7fd144161210 2023-10-19T10:38:25.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.435+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd14404e420 con 0x7fd144161210 2023-10-19T10:38:25.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.435+0000 7fd141ffb640 1 -- 172.21.15.50:0/2876339560 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd13c0570a0 con 0x7fd144161210 2023-10-19T10:38:25.439 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:38:25.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.437+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd13006a050 msgr2=0x7fd13006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.437+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd13006a050 0x7fd13006c510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd134036000 tx=0x7fd134008030 comp rx=0 tx=0).stop 2023-10-19T10:38:25.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.437+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 msgr2=0x7fd14414b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.437+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 0x7fd14414b480 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b9e0 tx=0x7fd13c00bea0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 shutdown_connections 2023-10-19T10:38:25.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd13006a050 0x7fd13006c510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd144161210 0x7fd14414b480 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1440b38f0 0x7fd14414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 --2- 172.21.15.50:0/2876339560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1440b3090 0x7fd14414d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 >> 172.21.15.50:0/2876339560 conn(0x7fd14409b960 msgr2=0x7fd14415e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:25.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 shutdown_connections 2023-10-19T10:38:25.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.438+0000 7fd14b3d0640 1 -- 172.21.15.50:0/2876339560 wait complete. 2023-10-19T10:38:25.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:38:25.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.557+0000 7f75b9228640 1 Processor -- start 2023-10-19T10:38:25.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.557+0000 7f75b9228640 1 -- start start 2023-10-19T10:38:25.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b9228640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b40d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b9228640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f75b40b3090 0x7f75b4160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b9228640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40af250 con 0x7f75b40b3cd0 2023-10-19T10:38:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b9228640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b40b0190 con 0x7f75b40b3090 2023-10-19T10:38:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b9228640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40af510 con 0x7f75b40b38f0 2023-10-19T10:38:25.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b259c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f75b40b3090 0x7f75b4160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b359e640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f75b40b3cd0 0x7f75b40b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:50270/0 (socket says 172.21.15.50:50270) 2023-10-19T10:38:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b359e640 1 -- 172.21.15.50:0/3933765358 learned_addr learned my addr 172.21.15.50:0/3933765358 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.558+0000 7f75b2d9d640 1 --2- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b40d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.559+0000 7f75b259c640 1 -- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 msgr2=0x7f75b40d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:25.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.559+0000 7f75b259c640 1 --2- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b40d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.559+0000 7f75b259c640 1 -- 172.21.15.50:0/3933765358 >> v1:172.21.15.183:6789/0 conn(0x7f75b40b3cd0 legacy=0x7f75b40b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.559+0000 7f75b259c640 1 -- 172.21.15.50:0/3933765358 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40af7f0 con 0x7f75b40b3090 2023-10-19T10:38:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.559+0000 7f75b259c640 1 --2- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4160b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f759c0097a0 tx=0x7f759c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37f2f1737e00cba server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b1d9b640 1 -- 172.21.15.50:0/3933765358 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c031830 con 0x7f75b40b3090 2023-10-19T10:38:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b1d9b640 1 -- 172.21.15.50:0/3933765358 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c031990 con 0x7f75b40b3090 2023-10-19T10:38:25.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b1d9b640 1 -- 172.21.15.50:0/3933765358 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c031d20 con 0x7f75b40b3090 2023-10-19T10:38:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 -- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 msgr2=0x7f75b4160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 --2- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4160b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f759c0097a0 tx=0x7f759c02b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 -- 172.21.15.50:0/3933765358 shutdown_connections 2023-10-19T10:38:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 --2- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4160b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 --2- 172.21.15.50:0/3933765358 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b40d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 -- 172.21.15.50:0/3933765358 >> 172.21.15.50:0/3933765358 conn(0x7f75b409b960 msgr2=0x7f75b40d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 -- 172.21.15.50:0/3933765358 shutdown_connections 2023-10-19T10:38:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.560+0000 7f75b9228640 1 -- 172.21.15.50:0/3933765358 wait complete. 2023-10-19T10:38:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 Processor -- start 2023-10-19T10:38:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 -- start start 2023-10-19T10:38:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 0x7f75b415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b415a890 con 0x7f75b40b3cd0 2023-10-19T10:38:25.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4159ca0 con 0x7f75b40b3090 2023-10-19T10:38:25.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.562+0000 7f75b9228640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b414b530 con 0x7f75b40b38f0 2023-10-19T10:38:25.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 0x7f75b415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 0x7f75b415be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42294/0 (socket says 172.21.15.50:42294) 2023-10-19T10:38:25.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 -- 172.21.15.50:0/3134449286 learned_addr learned my addr 172.21.15.50:0/3134449286 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:25.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b259c640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b2d9d640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 -- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 msgr2=0x7f75b415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b415fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 -- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 msgr2=0x7f75b4155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 -- 172.21.15.50:0/3134449286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009400 con 0x7f75b40b3cd0 2023-10-19T10:38:25.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b359e640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 0x7f75b415be30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f75a400ca80 tx=0x7f75a400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400ac30 con 0x7f75b40b3cd0 2023-10-19T10:38:25.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a400ad90 con 0x7f75b40b3cd0 2023-10-19T10:38:25.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b4159ef0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4012600 con 0x7f75b40b3cd0 2023-10-19T10:38:25.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.563+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b415a4b0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.564+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b415a9d0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.565+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f75a4007070 con 0x7f75b40b3cd0 2023-10-19T10:38:25.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.565+0000 7f75ab7fe640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f759406a080 0x7f759406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.565+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f75a40881e0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.565+0000 7f75b2d9d640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f759406a080 0x7f759406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.566+0000 7f75b2d9d640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f759406a080 0x7f759406c540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009b30 tx=0x7f75ac009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.570+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f75b415a9d0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.748+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75b415a9d0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.748+0000 7f75ab7fe640 1 -- 172.21.15.50:0/3134449286 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75b415a9d0 con 0x7f75b40b3cd0 2023-10-19T10:38:25.751 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:38:25.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f759406a080 msgr2=0x7f759406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f759406a080 0x7f759406c540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009b30 tx=0x7f75ac009110 comp rx=0 tx=0).stop 2023-10-19T10:38:25.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 msgr2=0x7f75b415be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 0x7f75b415be30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f75a400ca80 tx=0x7f75a400cf40 comp rx=0 tx=0).stop 2023-10-19T10:38:25.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 shutdown_connections 2023-10-19T10:38:25.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f759406a080 0x7f759406c540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75b40b3cd0 0x7f75b415be30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75b40b38f0 0x7f75b415fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 --2- 172.21.15.50:0/3134449286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75b40b3090 0x7f75b4155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 >> 172.21.15.50:0/3134449286 conn(0x7f75b409b960 msgr2=0x7f75b40b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:25.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 shutdown_connections 2023-10-19T10:38:25.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.750+0000 7f75b9228640 1 -- 172.21.15.50:0/3134449286 wait complete. 2023-10-19T10:38:25.763 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-19T10:38:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.867+0000 7f4746f6e640 1 Processor -- start 2023-10-19T10:38:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.867+0000 7f4746f6e640 1 -- start start 2023-10-19T10:38:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4746f6e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4746f6e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f474015c950 0x7f47400cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4746f6e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f474006b760 0x7f47400d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4746f6e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47400cde60 con 0x7f474015c950 2023-10-19T10:38:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4746f6e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474006aa30 con 0x7f474006b760 2023-10-19T10:38:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4746f6e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474006b2d0 con 0x7f474015c570 2023-10-19T10:38:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f474015c950 0x7f47400cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f474015c950 0x7f47400cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42310/0 (socket says 172.21.15.50:42310) 2023-10-19T10:38:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 -- 172.21.15.50:0/3242620138 learned_addr learned my addr 172.21.15.50:0/3242620138 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f474550b640 1 --2- 172.21.15.50:0/3242620138 >> v2:172.21.15.183:3300/0 conn(0x7f474006b760 0x7f47400d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f4744d0a640 1 --2- 172.21.15.50:0/3242620138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 -- 172.21.15.50:0/3242620138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 msgr2=0x7f47400d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 --2- 172.21.15.50:0/3242620138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 -- 172.21.15.50:0/3242620138 >> v2:172.21.15.183:3300/0 conn(0x7f474006b760 msgr2=0x7f47400d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:25.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 --2- 172.21.15.50:0/3242620138 >> v2:172.21.15.183:3300/0 conn(0x7f474006b760 0x7f47400d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.868+0000 7f473ffff640 1 -- 172.21.15.50:0/3242620138 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400ce090 con 0x7f474015c950 2023-10-19T10:38:25.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.869+0000 7f473ffff640 1 --2- 172.21.15.50:0/3242620138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f47400cd920 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f47401615b0 tx=0x7f473002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c10767489722d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.869+0000 7f473f7fe640 1 -- 172.21.15.50:0/3242620138 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473002c9e0 con 0x7f474015c950 2023-10-19T10:38:25.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.869+0000 7f473f7fe640 1 -- 172.21.15.50:0/3242620138 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473002cb40 con 0x7f474015c950 2023-10-19T10:38:25.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.869+0000 7f473f7fe640 1 -- 172.21.15.50:0/3242620138 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300343f0 con 0x7f474015c950 2023-10-19T10:38:25.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 -- 172.21.15.50:0/3242620138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 msgr2=0x7f47400cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3242620138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f47400cd920 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f47401615b0 tx=0x7f473002be40 comp rx=0 tx=0).stop 2023-10-19T10:38:25.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 -- 172.21.15.50:0/3242620138 shutdown_connections 2023-10-19T10:38:25.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3242620138 >> v2:172.21.15.183:3300/0 conn(0x7f474006b760 0x7f47400d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3242620138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f47400cd920 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3242620138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 -- 172.21.15.50:0/3242620138 >> 172.21.15.50:0/3242620138 conn(0x7f4740089ff0 msgr2=0x7f474009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:25.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 -- 172.21.15.50:0/3242620138 shutdown_connections 2023-10-19T10:38:25.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.870+0000 7f4746f6e640 1 -- 172.21.15.50:0/3242620138 wait complete. 2023-10-19T10:38:25.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.871+0000 7f4746f6e640 1 Processor -- start 2023-10-19T10:38:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.871+0000 7f4746f6e640 1 -- start start 2023-10-19T10:38:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4746f6e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474006b760 0x7f47400cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4746f6e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4746f6e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f4740157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4746f6e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c7cb0 con 0x7f474015c950 2023-10-19T10:38:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4746f6e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c70a0 con 0x7f474006b760 2023-10-19T10:38:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4746f6e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c7380 con 0x7f474015c570 2023-10-19T10:38:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f474550b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f4740157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f474550b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f4740157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42318/0 (socket says 172.21.15.50:42318) 2023-10-19T10:38:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f474550b640 1 -- 172.21.15.50:0/3475449882 learned_addr learned my addr 172.21.15.50:0/3475449882 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f473ffff640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.872+0000 7f4744d0a640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474006b760 0x7f47400cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f474550b640 1 -- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 msgr2=0x7f47400b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f474550b640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f474550b640 1 -- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474006b760 msgr2=0x7f47400cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f474550b640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474006b760 0x7f47400cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f474550b640 1 -- 172.21.15.50:0/3475449882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730009b30 con 0x7f474015c950 2023-10-19T10:38:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f4744d0a640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474006b760 0x7f47400cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f474550b640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f4740157b90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f472800d500 tx=0x7f472800d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728014620 con 0x7f474015c950 2023-10-19T10:38:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47280147a0 con 0x7f474015c950 2023-10-19T10:38:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47400c7660 con 0x7f474015c950 2023-10-19T10:38:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728014b50 con 0x7f474015c950 2023-10-19T10:38:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.873+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47400c8140 con 0x7f474015c950 2023-10-19T10:38:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.874+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47400c7e20 con 0x7f474015c950 2023-10-19T10:38:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.874+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4728014cb0 con 0x7f474015c950 2023-10-19T10:38:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.875+0000 7f473d7fa640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f472006a050 0x7f472006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.875+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4728088560 con 0x7f474015c950 2023-10-19T10:38:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.875+0000 7f4744d0a640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f472006a050 0x7f472006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.876+0000 7f4744d0a640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f472006a050 0x7f472006c510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f473800a9b0 tx=0x7f4738008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:25.879+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4728053e70 con 0x7f474015c950 2023-10-19T10:38:25.905 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:38:25.908 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:38:26.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.057+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4740073990 con 0x7f474015c950 2023-10-19T10:38:26.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.057+0000 7f473d7fa640 1 -- 172.21.15.50:0/3475449882 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4728057490 con 0x7f474015c950 2023-10-19T10:38:26.060 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:26.061 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2023-10-19T10:38:26.061 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:26.061 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2023-10-19T10:38:26.061 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2023-10-19T10:38:26.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:26.062 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:38:26.062 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 2, 2023-10-19T10:38:26.062 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": "rbd" 2023-10-19T10:38:26.062 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:26.063 INFO:teuthology.orchestra.run.smithi050.stdout:] 2023-10-19T10:38:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f472006a050 msgr2=0x7f472006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f472006a050 0x7f472006c510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f473800a9b0 tx=0x7f4738008040 comp rx=0 tx=0).stop 2023-10-19T10:38:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 msgr2=0x7f4740157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f4740157b90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f472800d500 tx=0x7f472800d9c0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 shutdown_connections 2023-10-19T10:38:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f472006a050 0x7f472006c510 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f474015c950 0x7f4740157b90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474015c570 0x7f47400b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 --2- 172.21.15.50:0/3475449882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474006b760 0x7f47400cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 >> 172.21.15.50:0/3475449882 conn(0x7f4740089ff0 msgr2=0x7f47400b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 shutdown_connections 2023-10-19T10:38:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.059+0000 7f4746f6e640 1 -- 172.21.15.50:0/3475449882 wait complete. 2023-10-19T10:38:26.072 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-19T10:38:26.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-19T10:38:26.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.180+0000 7f37fe628640 1 Processor -- start 2023-10-19T10:38:26.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.180+0000 7f37fe628640 1 -- start start 2023-10-19T10:38:26.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37fe628640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37f815c620 0x7f37f80b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37fe628640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37f80b82f0 0x7f37f80cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37fe628640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f80cda30 con 0x7f37f80b55c0 2023-10-19T10:38:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37fe628640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f80ccc10 con 0x7f37f80b82f0 2023-10-19T10:38:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37fe628640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f8161110 con 0x7f37f815c620 2023-10-19T10:38:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37f77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37f80b82f0 0x7f37f80cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37f77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37f80b82f0 0x7f37f80cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42330/0 (socket says 172.21.15.50:42330) 2023-10-19T10:38:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.181+0000 7f37f77fe640 1 -- 172.21.15.50:0/1823092772 learned_addr learned my addr 172.21.15.50:0/1823092772 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.182+0000 7f37f77fe640 1 -- 172.21.15.50:0/1823092772 >> v2:172.21.15.183:3300/0 conn(0x7f37f815c620 msgr2=0x7f37f80b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.182+0000 7f37f77fe640 1 --2- 172.21.15.50:0/1823092772 >> v2:172.21.15.183:3300/0 conn(0x7f37f815c620 0x7f37f80b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.182+0000 7f37f77fe640 1 -- 172.21.15.50:0/1823092772 >> v1:172.21.15.183:6789/0 conn(0x7f37f80b55c0 legacy=0x7f37f80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.182+0000 7f37f77fe640 1 -- 172.21.15.50:0/1823092772 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f80cce20 con 0x7f37f80b82f0 2023-10-19T10:38:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.182+0000 7f37f77fe640 1 --2- 172.21.15.50:0/1823092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f80cf3f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f37f80cefa0 tx=0x7f37e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867b6cd788621248 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37f6ffd640 1 -- 172.21.15.50:0/1823092772 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8031870 con 0x7f37f80b82f0 2023-10-19T10:38:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37f6ffd640 1 -- 172.21.15.50:0/1823092772 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e80319d0 con 0x7f37f80b82f0 2023-10-19T10:38:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37f6ffd640 1 -- 172.21.15.50:0/1823092772 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8031d60 con 0x7f37f80b82f0 2023-10-19T10:38:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 -- 172.21.15.50:0/1823092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 msgr2=0x7f37f80cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 --2- 172.21.15.50:0/1823092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f80cf3f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f37f80cefa0 tx=0x7f37e802b910 comp rx=0 tx=0).stop 2023-10-19T10:38:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 -- 172.21.15.50:0/1823092772 shutdown_connections 2023-10-19T10:38:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 --2- 172.21.15.50:0/1823092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f80cf3f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 --2- 172.21.15.50:0/1823092772 >> v2:172.21.15.183:3300/0 conn(0x7f37f815c620 0x7f37f80b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 -- 172.21.15.50:0/1823092772 >> 172.21.15.50:0/1823092772 conn(0x7f37f8071110 msgr2=0x7f37f80714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 -- 172.21.15.50:0/1823092772 shutdown_connections 2023-10-19T10:38:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.183+0000 7f37fe628640 1 -- 172.21.15.50:0/1823092772 wait complete. 2023-10-19T10:38:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.184+0000 7f37fe628640 1 Processor -- start 2023-10-19T10:38:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.184+0000 7f37fe628640 1 -- start start 2023-10-19T10:38:26.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fe628640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f80b55c0 0x7f37f815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fe628640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fe628640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f815c620 0x7f37f80b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fe628640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f80cc430 con 0x7f37f80b82f0 2023-10-19T10:38:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fe628640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f815ca00 con 0x7f37f815c620 2023-10-19T10:38:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fe628640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f815cc10 con 0x7f37f80b55c0 2023-10-19T10:38:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37f77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37f77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f8150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42340/0 (socket says 172.21.15.50:42340) 2023-10-19T10:38:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37f77fe640 1 -- 172.21.15.50:0/2339165891 learned_addr learned my addr 172.21.15.50:0/2339165891 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37fcbc5640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f815c620 0x7f37f80b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.185+0000 7f37f7fff640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f80b55c0 0x7f37f815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f77fe640 1 -- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f80b55c0 msgr2=0x7f37f815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f77fe640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f80b55c0 0x7f37f815f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f77fe640 1 -- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f815c620 msgr2=0x7f37f80b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f77fe640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f815c620 0x7f37f80b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f77fe640 1 -- 172.21.15.50:0/2339165891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80094d0 con 0x7f37f80b82f0 2023-10-19T10:38:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37fcbc5640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f815c620 0x7f37f80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f7fff640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f80b55c0 0x7f37f815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f77fe640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f8150ad0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000c00 tx=0x7f37e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e803ad10 con 0x7f37f80b82f0 2023-10-19T10:38:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e803ae70 con 0x7f37f80b82f0 2023-10-19T10:38:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f815cef0 con 0x7f37f80b82f0 2023-10-19T10:38:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8044610 con 0x7f37f80b82f0 2023-10-19T10:38:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.186+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f815d4b0 con 0x7f37f80b82f0 2023-10-19T10:38:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.187+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f804e420 con 0x7f37f80b82f0 2023-10-19T10:38:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.187+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f37e804d450 con 0x7f37f80b82f0 2023-10-19T10:38:26.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.188+0000 7f37f4ff9640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37d806a080 0x7f37d806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.188+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f37e802d040 con 0x7f37f80b82f0 2023-10-19T10:38:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.188+0000 7f37f7fff640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37d806a080 0x7f37d806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.189+0000 7f37f7fff640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37d806a080 0x7f37d806c540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37e0006fd0 tx=0x7f37e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.192+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f37e80763f0 con 0x7f37f80b82f0 2023-10-19T10:38:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.368+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f37f806b6f0 con 0x7f37f80b82f0 2023-10-19T10:38:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.561+0000 7f37f4ff9640 1 -- 172.21.15.50:0/2339165891 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37e8079a10 con 0x7f37f80b82f0 2023-10-19T10:38:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2023-10-19T10:38:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37d806a080 msgr2=0x7f37d806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37d806a080 0x7f37d806c540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37e0006fd0 tx=0x7f37e0008040 comp rx=0 tx=0).stop 2023-10-19T10:38:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 msgr2=0x7f37f8150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f8150ad0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000c00 tx=0x7f37e80046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 shutdown_connections 2023-10-19T10:38:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37d806a080 0x7f37d806c540 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f815c620 0x7f37f80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37f80b82f0 0x7f37f8150ad0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 --2- 172.21.15.50:0/2339165891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f80b55c0 0x7f37f815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 >> 172.21.15.50:0/2339165891 conn(0x7f37f8071110 msgr2=0x7f37f80d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 shutdown_connections 2023-10-19T10:38:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.563+0000 7f37fe628640 1 -- 172.21.15.50:0/2339165891 wait complete. 2023-10-19T10:38:26.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-19T10:38:26.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.690+0000 7fe8604e9640 1 Processor -- start 2023-10-19T10:38:26.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.691+0000 7fe8604e9640 1 -- start start 2023-10-19T10:38:26.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.691+0000 7fe8604e9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe8580b38f0 0x7fe8580d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.691+0000 7fe8604e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3090 0x7fe858160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.691+0000 7fe8604e9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580af250 con 0x7fe8580b3cd0 2023-10-19T10:38:26.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.691+0000 7fe8604e9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580b0190 con 0x7fe8580b38f0 2023-10-19T10:38:26.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.691+0000 7fe8604e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580af510 con 0x7fe8580b3090 2023-10-19T10:38:26.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe8580b38f0 0x7fe8580d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe8580b38f0 0x7fe8580d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41802/0 (socket says 172.21.15.50:41802) 2023-10-19T10:38:26.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 -- 172.21.15.50:0/1358108688 learned_addr learned my addr 172.21.15.50:0/1358108688 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:26.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85da84640 1 --2- 172.21.15.50:0/1358108688 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3090 0x7fe858160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 -- 172.21.15.50:0/1358108688 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3090 msgr2=0x7fe858160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:26.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 --2- 172.21.15.50:0/1358108688 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3090 0x7fe858160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 -- 172.21.15.50:0/1358108688 >> v1:172.21.15.183:6789/0 conn(0x7fe8580b3cd0 legacy=0x7fe8580b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.692+0000 7fe85e285640 1 -- 172.21.15.50:0/1358108688 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580af7f0 con 0x7fe8580b38f0 2023-10-19T10:38:26.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe85e285640 1 --2- 172.21.15.50:0/1358108688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b38f0 0x7fe8580d30e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe854009850 tx=0x7fe85402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f08634fa340c1bd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:26.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe85d283640 1 -- 172.21.15.50:0/1358108688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854004230 con 0x7fe8580b38f0 2023-10-19T10:38:26.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe85d283640 1 -- 172.21.15.50:0/1358108688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854004390 con 0x7fe8580b38f0 2023-10-19T10:38:26.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe85d283640 1 -- 172.21.15.50:0/1358108688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854003810 con 0x7fe8580b38f0 2023-10-19T10:38:26.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe8604e9640 1 -- 172.21.15.50:0/1358108688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b38f0 msgr2=0x7fe8580d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe8604e9640 1 --2- 172.21.15.50:0/1358108688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b38f0 0x7fe8580d30e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe854009850 tx=0x7fe85402b960 comp rx=0 tx=0).stop 2023-10-19T10:38:26.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe8604e9640 1 -- 172.21.15.50:0/1358108688 shutdown_connections 2023-10-19T10:38:26.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe8604e9640 1 --2- 172.21.15.50:0/1358108688 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3090 0x7fe858160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe8604e9640 1 --2- 172.21.15.50:0/1358108688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b38f0 0x7fe8580d30e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.693+0000 7fe8604e9640 1 -- 172.21.15.50:0/1358108688 >> 172.21.15.50:0/1358108688 conn(0x7fe85809b960 msgr2=0x7fe8580d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:26.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.694+0000 7fe8604e9640 1 -- 172.21.15.50:0/1358108688 shutdown_connections 2023-10-19T10:38:26.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.694+0000 7fe8604e9640 1 -- 172.21.15.50:0/1358108688 wait complete. 2023-10-19T10:38:26.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 Processor -- start 2023-10-19T10:38:26.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 -- start start 2023-10-19T10:38:26.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 0x7fe8580ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8580b38f0 0x7fe858156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3cd0 0x7fe8580cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858156e60 con 0x7fe8580b3090 2023-10-19T10:38:26.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815d900 con 0x7fe8580b38f0 2023-10-19T10:38:26.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe8604e9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815d5d0 con 0x7fe8580b3cd0 2023-10-19T10:38:26.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe85e285640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 0x7fe8580ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.695+0000 7fe85e285640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 0x7fe8580ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41804/0 (socket says 172.21.15.50:41804) 2023-10-19T10:38:26.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 -- 172.21.15.50:0/2991314311 learned_addr learned my addr 172.21.15.50:0/2991314311 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:26.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85ea86640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3cd0 0x7fe8580cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85da84640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8580b38f0 0x7fe858156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 -- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3cd0 msgr2=0x7fe8580cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3cd0 0x7fe8580cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 -- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8580b38f0 msgr2=0x7fe858156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:26.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8580b38f0 0x7fe858156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:26.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 -- 172.21.15.50:0/2991314311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe854009460 con 0x7fe8580b3090 2023-10-19T10:38:26.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe85e285640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 0x7fe8580ca2a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe85402b9a0 tx=0x7fe8540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:26.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85403e070 con 0x7fe8580b3090 2023-10-19T10:38:26.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854004500 con 0x7fe8580b3090 2023-10-19T10:38:26.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8580ca8a0 con 0x7fe8580b3090 2023-10-19T10:38:26.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854003e10 con 0x7fe8580b3090 2023-10-19T10:38:26.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.696+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580b52c0 con 0x7fe8580b3090 2023-10-19T10:38:26.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.697+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85804e420 con 0x7fe8580b3090 2023-10-19T10:38:26.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.698+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe854006040 con 0x7fe8580b3090 2023-10-19T10:38:26.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.698+0000 7fe84effd640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe83c06a080 0x7fe83c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:26.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.698+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fe8540aa280 con 0x7fe8580b3090 2023-10-19T10:38:26.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.699+0000 7fe85da84640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe83c06a080 0x7fe83c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:26.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.699+0000 7fe85da84640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe83c06a080 0x7fe83c06c540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe8580cdf00 tx=0x7fe848008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:26.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.703+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe854075ac0 con 0x7fe8580b3090 2023-10-19T10:38:26.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:26.879+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8580b86b0 con 0x7fe8580b3090 2023-10-19T10:38:27.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.571+0000 7fe84effd640 1 -- 172.21.15.50:0/2991314311 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8540790e0 con 0x7fe8580b3090 2023-10-19T10:38:27.574 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2023-10-19T10:38:27.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.580+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe83c06a080 msgr2=0x7fe83c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe83c06a080 0x7fe83c06c540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe8580cdf00 tx=0x7fe848008040 comp rx=0 tx=0).stop 2023-10-19T10:38:27.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 msgr2=0x7fe8580ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 0x7fe8580ca2a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe85402b9a0 tx=0x7fe8540046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 shutdown_connections 2023-10-19T10:38:27.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe83c06a080 0x7fe83c06c540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8580b3cd0 0x7fe8580cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8580b38f0 0x7fe858156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 --2- 172.21.15.50:0/2991314311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8580b3090 0x7fe8580ca2a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.581+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 >> 172.21.15.50:0/2991314311 conn(0x7fe85809b960 msgr2=0x7fe8580d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.583+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 shutdown_connections 2023-10-19T10:38:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.583+0000 7fe8604e9640 1 -- 172.21.15.50:0/2991314311 wait complete. 2023-10-19T10:38:27.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-19T10:38:27.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.717+0000 7f0c245d2640 1 Processor -- start 2023-10-19T10:38:27.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.717+0000 7f0c245d2640 1 -- start start 2023-10-19T10:38:27.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c245d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c15c740 0x7f0c1c0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:27.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c245d2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b5510 con 0x7f0c1c0b4750 2023-10-19T10:38:27.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c245d2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0d0bd0 con 0x7f0c1c0d0750 2023-10-19T10:38:27.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c245d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0d0eb0 con 0x7f0c1c15c740 2023-10-19T10:38:27.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c22b6f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f0c1c0b4750 0x7f0c1c0b59f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58852/0 (socket says 172.21.15.50:58852) 2023-10-19T10:38:27.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c22b6f640 1 -- 172.21.15.50:0/3832361313 learned_addr learned my addr 172.21.15.50:0/3832361313 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:27.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2236e640 1 --2- 172.21.15.50:0/3832361313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c15c740 0x7f0c1c0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:27.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100077889 0 0) 0x7f0c1c0b5510 con 0x7f0c1c0b4750 2023-10-19T10:38:27.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003520 con 0x7f0c1c0b4750 2023-10-19T10:38:27.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346489776 0 0) 0x7f0c1c0d0bd0 con 0x7f0c1c0d0750 2023-10-19T10:38:27.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c0b5510 con 0x7f0c1c0d0750 2023-10-19T10:38:27.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707798807 0 0) 0x7f0c08003520 con 0x7f0c1c0b4750 2023-10-19T10:38:27.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2236e640 1 -- 172.21.15.50:0/3832361313 >> v1:172.21.15.183:6789/0 conn(0x7f0c1c0d0750 legacy=0x7f0c1c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2236e640 1 -- 172.21.15.50:0/3832361313 >> v1:172.21.15.50:6789/0 conn(0x7f0c1c0b4750 legacy=0x7f0c1c0b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.718+0000 7f0c2236e640 1 -- 172.21.15.50:0/3832361313 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0d1190 con 0x7f0c1c15c740 2023-10-19T10:38:27.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.719+0000 7f0c2236e640 1 --2- 172.21.15.50:0/3832361313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c15c740 0x7f0c1c0d1c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c180097d0 tx=0x7f0c1802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd378ac4be3730 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:27.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.719+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140019f0 con 0x7f0c1c15c740 2023-10-19T10:38:27.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.719+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1802e050 con 0x7f0c1c15c740 2023-10-19T10:38:27.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.719+0000 7f0c2136c640 1 -- 172.21.15.50:0/3832361313 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18030710 con 0x7f0c1c15c740 2023-10-19T10:38:27.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 -- 172.21.15.50:0/3832361313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c15c740 msgr2=0x7f0c1c0d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3832361313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c15c740 0x7f0c1c0d1c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c180097d0 tx=0x7f0c1802b800 comp rx=0 tx=0).stop 2023-10-19T10:38:27.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 -- 172.21.15.50:0/3832361313 shutdown_connections 2023-10-19T10:38:27.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3832361313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c15c740 0x7f0c1c0d1c20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 -- 172.21.15.50:0/3832361313 >> 172.21.15.50:0/3832361313 conn(0x7f0c1c071320 msgr2=0x7f0c1c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:27.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 -- 172.21.15.50:0/3832361313 shutdown_connections 2023-10-19T10:38:27.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.720+0000 7f0c245d2640 1 -- 172.21.15.50:0/3832361313 wait complete. 2023-10-19T10:38:27.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.721+0000 7f0c245d2640 1 Processor -- start 2023-10-19T10:38:27.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.721+0000 7f0c245d2640 1 -- start start 2023-10-19T10:38:27.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c245d2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 0x7f0c1c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:27.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c245d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c0d0750 0x7f0c1c0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:27.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c245d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c1c15c740 0x7f0c1c1e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:27.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c245d2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c1f60f0 con 0x7f0c1c0b4750 2023-10-19T10:38:27.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c245d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c7140 con 0x7f0c1c15c740 2023-10-19T10:38:27.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c245d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c7420 con 0x7f0c1c0d0750 2023-10-19T10:38:27.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 0x7f0c1c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:27.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 0x7f0c1c15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41816/0 (socket says 172.21.15.50:41816) 2023-10-19T10:38:27.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 -- 172.21.15.50:0/3028659795 learned_addr learned my addr 172.21.15.50:0/3028659795 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:27.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c22b6f640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c1c15c740 0x7f0c1c1e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:27.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c21b6d640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c0d0750 0x7f0c1c0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:27.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 -- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c0d0750 msgr2=0x7f0c1c0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c0d0750 0x7f0c1c0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 -- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c1c15c740 msgr2=0x7f0c1c1e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:27.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c1c15c740 0x7f0c1c1e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:27.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.722+0000 7f0c2236e640 1 -- 172.21.15.50:0/3028659795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009400 con 0x7f0c1c0b4750 2023-10-19T10:38:27.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.723+0000 7f0c2236e640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 0x7f0c1c15b730 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:27.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.723+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1803c620 con 0x7f0c1c0b4750 2023-10-19T10:38:27.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.723+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1803c7a0 con 0x7f0c1c0b4750 2023-10-19T10:38:27.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.723+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c0c7700 con 0x7f0c1c0b4750 2023-10-19T10:38:27.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.723+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1803cb50 con 0x7f0c1c0b4750 2023-10-19T10:38:27.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.723+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c0c7cc0 con 0x7f0c1c0b4750 2023-10-19T10:38:27.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.724+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c04e420 con 0x7f0c1c0b4750 2023-10-19T10:38:27.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.724+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0c18039070 con 0x7f0c1c0b4750 2023-10-19T10:38:27.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.725+0000 7f0c12ffd640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0c0006a050 0x7f0c0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:27.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.725+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f0c180a9d50 con 0x7f0c1c0b4750 2023-10-19T10:38:27.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.725+0000 7f0c21b6d640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0c0006a050 0x7f0c0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:27.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.725+0000 7f0c21b6d640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0c0006a050 0x7f0c0006c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c1e5af0 tx=0x7f0c0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:27.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.729+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0c180ad030 con 0x7f0c1c0b4750 2023-10-19T10:38:27.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:27.940+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0c1c0b4e70 con 0x7f0c1c0b4750 2023-10-19T10:38:28.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.589+0000 7f0c12ffd640 1 -- 172.21.15.50:0/3028659795 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0c18078e20 con 0x7f0c1c0b4750 2023-10-19T10:38:28.596 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 6 and data pool 7 2023-10-19T10:38:28.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0c0006a050 msgr2=0x7f0c0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0c0006a050 0x7f0c0006c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c1e5af0 tx=0x7f0c0c009040 comp rx=0 tx=0).stop 2023-10-19T10:38:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 msgr2=0x7f0c1c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 0x7f0c1c15b730 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c180046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 shutdown_connections 2023-10-19T10:38:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0c0006a050 0x7f0c0006c510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c1c15c740 0x7f0c1c1e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c1c0d0750 0x7f0c1c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 --2- 172.21.15.50:0/3028659795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1c0b4750 0x7f0c1c15b730 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 >> 172.21.15.50:0/3028659795 conn(0x7f0c1c071320 msgr2=0x7f0c1c0cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 shutdown_connections 2023-10-19T10:38:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.597+0000 7f0c245d2640 1 -- 172.21.15.50:0/3028659795 wait complete. 2023-10-19T10:38:28.620 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:28.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 Processor -- start 2023-10-19T10:38:28.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 -- start start 2023-10-19T10:38:28.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373015c570 0x7f37300d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f373015c950 0x7f37300cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f373006b760 0x7f37300d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37300cde60 con 0x7f373006b760 2023-10-19T10:38:28.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373006aa30 con 0x7f373015c950 2023-10-19T10:38:28.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f37361b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373006b2d0 con 0x7f373015c570 2023-10-19T10:38:28.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.729+0000 7f372ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f373006b760 0x7f37300d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f373006b760 0x7f37300d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41830/0 (socket says 172.21.15.50:41830) 2023-10-19T10:38:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 -- 172.21.15.50:0/1164547534 learned_addr learned my addr 172.21.15.50:0/1164547534 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372effd640 1 --2- 172.21.15.50:0/1164547534 >> v2:172.21.15.183:3300/0 conn(0x7f373015c950 0x7f37300cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372f7fe640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373015c570 0x7f37300d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 -- 172.21.15.50:0/1164547534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373015c570 msgr2=0x7f37300d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373015c570 0x7f37300d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 -- 172.21.15.50:0/1164547534 >> v2:172.21.15.183:3300/0 conn(0x7f373015c950 msgr2=0x7f37300cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 --2- 172.21.15.50:0/1164547534 >> v2:172.21.15.183:3300/0 conn(0x7f373015c950 0x7f37300cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372ffff640 1 -- 172.21.15.50:0/1164547534 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300ce090 con 0x7f373006b760 2023-10-19T10:38:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.730+0000 7f372f7fe640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373015c570 0x7f37300d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f372ffff640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373006b760 0x7f37300d26a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f373008a610 tx=0x7f372802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3327219dcfbb2a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f372e7fc640 1 -- 172.21.15.50:0/1164547534 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372802c9e0 con 0x7f373006b760 2023-10-19T10:38:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f372e7fc640 1 -- 172.21.15.50:0/1164547534 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372802cb40 con 0x7f373006b760 2023-10-19T10:38:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f372e7fc640 1 -- 172.21.15.50:0/1164547534 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37280343f0 con 0x7f373006b760 2023-10-19T10:38:28.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 -- 172.21.15.50:0/1164547534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373006b760 msgr2=0x7f37300d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373006b760 0x7f37300d26a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f373008a610 tx=0x7f372802be40 comp rx=0 tx=0).stop 2023-10-19T10:38:28.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 -- 172.21.15.50:0/1164547534 shutdown_connections 2023-10-19T10:38:28.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373006b760 0x7f37300d26a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 --2- 172.21.15.50:0/1164547534 >> v2:172.21.15.183:3300/0 conn(0x7f373015c950 0x7f37300cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 --2- 172.21.15.50:0/1164547534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373015c570 0x7f37300d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 -- 172.21.15.50:0/1164547534 >> 172.21.15.50:0/1164547534 conn(0x7f3730089ff0 msgr2=0x7f373009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:28.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 -- 172.21.15.50:0/1164547534 shutdown_connections 2023-10-19T10:38:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.731+0000 7f37361b2640 1 -- 172.21.15.50:0/1164547534 wait complete. 2023-10-19T10:38:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.732+0000 7f37361b2640 1 Processor -- start 2023-10-19T10:38:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 -- start start 2023-10-19T10:38:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373006b760 0x7f37301f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 0x7f37300cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373015c950 0x7f3730160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3730150900 con 0x7f373015c570 2023-10-19T10:38:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373015eab0 con 0x7f373015c950 2023-10-19T10:38:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.733+0000 7f37361b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373015ed90 con 0x7f373006b760 2023-10-19T10:38:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 0x7f37300cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 0x7f37300cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41842/0 (socket says 172.21.15.50:41842) 2023-10-19T10:38:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 -- 172.21.15.50:0/2947819556 learned_addr learned my addr 172.21.15.50:0/2947819556 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372ffff640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373015c950 0x7f3730160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372f7fe640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373006b760 0x7f37301f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 -- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373006b760 msgr2=0x7f37301f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373006b760 0x7f37301f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 -- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373015c950 msgr2=0x7f3730160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373015c950 0x7f3730160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 -- 172.21.15.50:0/2947819556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728009b30 con 0x7f373015c570 2023-10-19T10:38:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f372effd640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 0x7f37300cab70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f372400e9b0 tx=0x7f372400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:28.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724009670 con 0x7f373015c570 2023-10-19T10:38:28.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372400c9a0 con 0x7f373015c570 2023-10-19T10:38:28.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373015f070 con 0x7f373015c570 2023-10-19T10:38:28.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372400cd50 con 0x7f373015c570 2023-10-19T10:38:28.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.734+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373015f630 con 0x7f373015c570 2023-10-19T10:38:28.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.735+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37300d2120 con 0x7f373015c570 2023-10-19T10:38:28.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.736+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3724013470 con 0x7f373015c570 2023-10-19T10:38:28.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.736+0000 7f3717fff640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f371c06a050 0x7f371c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:28.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.736+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f37240888d0 con 0x7f373015c570 2023-10-19T10:38:28.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.736+0000 7f372f7fe640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f371c06a050 0x7f371c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.737+0000 7f372f7fe640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f371c06a050 0x7f371c06c510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f37200098b0 tx=0x7f3720008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.741+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3724053f30 con 0x7f373015c570 2023-10-19T10:38:28.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.919+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3730073990 con 0x7f373015c570 2023-10-19T10:38:28.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.920+0000 7f3717fff640 1 -- 172.21.15.50:0/2947819556 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18477 (secure 0 0 0) 0x7f3724057550 con 0x7f373015c570 2023-10-19T10:38:28.923 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:28.924 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":41,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:38:27.942123+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:28.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f371c06a050 msgr2=0x7f371c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f371c06a050 0x7f371c06c510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f37200098b0 tx=0x7f3720008040 comp rx=0 tx=0).stop 2023-10-19T10:38:28.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 msgr2=0x7f37300cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:28.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 0x7f37300cab70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f372400e9b0 tx=0x7f372400ee70 comp rx=0 tx=0).stop 2023-10-19T10:38:28.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 shutdown_connections 2023-10-19T10:38:28.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f371c06a050 0x7f371c06c510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373015c950 0x7f3730160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f373015c570 0x7f37300cab70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 --2- 172.21.15.50:0/2947819556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f373006b760 0x7f37301f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:28.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 >> 172.21.15.50:0/2947819556 conn(0x7f3730089ff0 msgr2=0x7f373015bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:28.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 shutdown_connections 2023-10-19T10:38:28.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:28.922+0000 7f37361b2640 1 -- 172.21.15.50:0/2947819556 wait complete. 2023-10-19T10:38:28.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:29.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.041+0000 7fbbe2a2f640 1 Processor -- start 2023-10-19T10:38:29.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.041+0000 7fbbe2a2f640 1 -- start start 2023-10-19T10:38:29.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbe2a2f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc15c570 0x7fbbdc0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbe2a2f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbbdc15c950 0x7fbbdc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbe2a2f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbbdc06b760 0x7fbbdc0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbe2a2f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0cde60 con 0x7fbbdc15c950 2023-10-19T10:38:29.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbe2a2f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc06aa30 con 0x7fbbdc06b760 2023-10-19T10:38:29.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbe2a2f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc06b2d0 con 0x7fbbdc15c570 2023-10-19T10:38:29.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbdb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbbdc15c950 0x7fbbdc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbdb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbbdc15c950 0x7fbbdc0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41856/0 (socket says 172.21.15.50:41856) 2023-10-19T10:38:29.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.042+0000 7fbbdb7fe640 1 -- 172.21.15.50:0/300260006 learned_addr learned my addr 172.21.15.50:0/300260006 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:29.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdbfff640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc15c570 0x7fbbdc0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbe0fcc640 1 --2- 172.21.15.50:0/300260006 >> v2:172.21.15.183:3300/0 conn(0x7fbbdc06b760 0x7fbbdc0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdb7fe640 1 -- 172.21.15.50:0/300260006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc15c570 msgr2=0x7fbbdc0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdb7fe640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc15c570 0x7fbbdc0d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdb7fe640 1 -- 172.21.15.50:0/300260006 >> v2:172.21.15.183:3300/0 conn(0x7fbbdc06b760 msgr2=0x7fbbdc0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdb7fe640 1 --2- 172.21.15.50:0/300260006 >> v2:172.21.15.183:3300/0 conn(0x7fbbdc06b760 0x7fbbdc0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdb7fe640 1 -- 172.21.15.50:0/300260006 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0ce090 con 0x7fbbdc15c950 2023-10-19T10:38:29.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdbfff640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc15c570 0x7fbbdc0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:29.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdb7fe640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc0cd920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc1615b0 tx=0x7fbbcc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944bb4fb1b812536 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.043+0000 7fbbdaffd640 1 -- 172.21.15.50:0/300260006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0317d0 con 0x7fbbdc15c950 2023-10-19T10:38:29.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbdaffd640 1 -- 172.21.15.50:0/300260006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc031930 con 0x7fbbdc15c950 2023-10-19T10:38:29.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbdaffd640 1 -- 172.21.15.50:0/300260006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc031cc0 con 0x7fbbdc15c950 2023-10-19T10:38:29.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/300260006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 msgr2=0x7fbbdc0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc0cd920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc1615b0 tx=0x7fbbcc02b870 comp rx=0 tx=0).stop 2023-10-19T10:38:29.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/300260006 shutdown_connections 2023-10-19T10:38:29.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/300260006 >> v2:172.21.15.183:3300/0 conn(0x7fbbdc06b760 0x7fbbdc0d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc0cd920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/300260006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc15c570 0x7fbbdc0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/300260006 >> 172.21.15.50:0/300260006 conn(0x7fbbdc089ff0 msgr2=0x7fbbdc09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:29.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/300260006 shutdown_connections 2023-10-19T10:38:29.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.044+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/300260006 wait complete. 2023-10-19T10:38:29.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.045+0000 7fbbe2a2f640 1 Processor -- start 2023-10-19T10:38:29.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.045+0000 7fbbe2a2f640 1 -- start start 2023-10-19T10:38:29.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe2a2f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc06b760 0x7fbbdc0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe2a2f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbdc15c570 0x7fbbdc149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe2a2f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe2a2f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc14af20 con 0x7fbbdc15c950 2023-10-19T10:38:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe2a2f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc14a5d0 con 0x7fbbdc15c570 2023-10-19T10:38:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe2a2f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc14a8a0 con 0x7fbbdc06b760 2023-10-19T10:38:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe0fcc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe0fcc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc14a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41868/0 (socket says 172.21.15.50:41868) 2023-10-19T10:38:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbe0fcc640 1 -- 172.21.15.50:0/1761596495 learned_addr learned my addr 172.21.15.50:0/1761596495 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbdbfff640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc06b760 0x7fbbdc0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.046+0000 7fbbdb7fe640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbdc15c570 0x7fbbdc149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe0fcc640 1 -- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc06b760 msgr2=0x7fbbdc0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe0fcc640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc06b760 0x7fbbdc0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe0fcc640 1 -- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbdc15c570 msgr2=0x7fbbdc149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe0fcc640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbdc15c570 0x7fbbdc149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe0fcc640 1 -- 172.21.15.50:0/1761596495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0094d0 con 0x7fbbdc15c950 2023-10-19T10:38:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe0fcc640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc14a090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c9a0 tx=0x7fbbd400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400a990 con 0x7fbbdc15c950 2023-10-19T10:38:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd400aaf0 con 0x7fbbdc15c950 2023-10-19T10:38:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc15fa20 con 0x7fbbdc15c950 2023-10-19T10:38:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4011410 con 0x7fbbdc15c950 2023-10-19T10:38:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.047+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc14ab40 con 0x7fbbdc15c950 2023-10-19T10:38:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.048+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbdc073990 con 0x7fbbdc15c950 2023-10-19T10:38:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.049+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbbd4011570 con 0x7fbbdc15c950 2023-10-19T10:38:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.049+0000 7fbbd8ff9640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbbc806a050 0x7fbbc806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.049+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbbd4088390 con 0x7fbbdc15c950 2023-10-19T10:38:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.049+0000 7fbbdbfff640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbbc806a050 0x7fbbc806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.050+0000 7fbbdbfff640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbbc806a050 0x7fbbc806c510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0006fd0 tx=0x7fbbd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.053+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbbd4053a80 con 0x7fbbdc15c950 2023-10-19T10:38:29.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.228+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbdc0d3480 con 0x7fbbdc15c950 2023-10-19T10:38:29.232 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.229+0000 7fbbd8ff9640 1 -- 172.21.15.50:0/1761596495 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18477 (secure 0 0 0) 0x7fbbd40570a0 con 0x7fbbdc15c950 2023-10-19T10:38:29.233 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:29.233 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":41,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:38:27.942123+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:29.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbbc806a050 msgr2=0x7fbbc806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbbc806a050 0x7fbbc806c510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0006fd0 tx=0x7fbbd0008040 comp rx=0 tx=0).stop 2023-10-19T10:38:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 msgr2=0x7fbbdc14a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc14a090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c9a0 tx=0x7fbbd400ce60 comp rx=0 tx=0).stop 2023-10-19T10:38:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 shutdown_connections 2023-10-19T10:38:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbbc806a050 0x7fbbc806c510 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbdc15c950 0x7fbbdc14a090 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbdc15c570 0x7fbbdc149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 --2- 172.21.15.50:0/1761596495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbdc06b760 0x7fbbdc0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.231+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 >> 172.21.15.50:0/1761596495 conn(0x7fbbdc089ff0 msgr2=0x7fbbdc0af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.232+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 shutdown_connections 2023-10-19T10:38:29.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.232+0000 7fbbe2a2f640 1 -- 172.21.15.50:0/1761596495 wait complete. 2023-10-19T10:38:29.245 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-19T10:38:29.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.351+0000 7f9f2c241640 1 Processor -- start 2023-10-19T10:38:29.354 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.351+0000 7f9f2c241640 1 -- start start 2023-10-19T10:38:29.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f2c241640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9f24069a70 0x7f9f240c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f2c241640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f240cb6f0 0x7f9f2406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f2c241640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f240ca120 con 0x7f9f24069e50 2023-10-19T10:38:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f2c241640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f24071b90 con 0x7f9f240cb6f0 2023-10-19T10:38:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f2c241640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f24071310 con 0x7f9f24069a70 2023-10-19T10:38:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f297dc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f240cb6f0 0x7f9f2406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f297dc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f240cb6f0 0x7f9f2406dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41878/0 (socket says 172.21.15.50:41878) 2023-10-19T10:38:29.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f297dc640 1 -- 172.21.15.50:0/2459919692 learned_addr learned my addr 172.21.15.50:0/2459919692 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f2a7de640 1 --1- 172.21.15.50:0/2459919692 >> v1:172.21.15.50:6789/0 conn(0x7f9f24069e50 0x7f9f2406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58868/0 (socket says 172.21.15.50:58868) 2023-10-19T10:38:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.352+0000 7f9f29fdd640 1 --2- 172.21.15.50:0/2459919692 >> v2:172.21.15.183:3300/0 conn(0x7f9f24069a70 0x7f9f240c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.353+0000 7f9f297dc640 1 -- 172.21.15.50:0/2459919692 >> v2:172.21.15.183:3300/0 conn(0x7f9f24069a70 msgr2=0x7f9f240c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.353+0000 7f9f297dc640 1 --2- 172.21.15.50:0/2459919692 >> v2:172.21.15.183:3300/0 conn(0x7f9f24069a70 0x7f9f240c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.353+0000 7f9f297dc640 1 -- 172.21.15.50:0/2459919692 >> v1:172.21.15.50:6789/0 conn(0x7f9f24069e50 legacy=0x7f9f2406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.353+0000 7f9f297dc640 1 -- 172.21.15.50:0/2459919692 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240715d0 con 0x7f9f240cb6f0 2023-10-19T10:38:29.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.353+0000 7f9f28fdb640 1 -- 172.21.15.50:0/2459919692 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389302804 0 0) 0x7f9f240ca120 con 0x7f9f24069e50 2023-10-19T10:38:29.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.353+0000 7f9f297dc640 1 --2- 172.21.15.50:0/2459919692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f240cb6f0 0x7f9f2406dc00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f240731e0 tx=0x7f9f1802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dca9ce70b07a91 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f28fdb640 1 -- 172.21.15.50:0/2459919692 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180317e0 con 0x7f9f240cb6f0 2023-10-19T10:38:29.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f28fdb640 1 -- 172.21.15.50:0/2459919692 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18031940 con 0x7f9f240cb6f0 2023-10-19T10:38:29.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f28fdb640 1 -- 172.21.15.50:0/2459919692 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18031cd0 con 0x7f9f240cb6f0 2023-10-19T10:38:29.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 -- 172.21.15.50:0/2459919692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f240cb6f0 msgr2=0x7f9f2406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 --2- 172.21.15.50:0/2459919692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f240cb6f0 0x7f9f2406dc00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f240731e0 tx=0x7f9f1802b880 comp rx=0 tx=0).stop 2023-10-19T10:38:29.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 -- 172.21.15.50:0/2459919692 shutdown_connections 2023-10-19T10:38:29.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 --2- 172.21.15.50:0/2459919692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f240cb6f0 0x7f9f2406dc00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 --2- 172.21.15.50:0/2459919692 >> v2:172.21.15.183:3300/0 conn(0x7f9f24069a70 0x7f9f240c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 -- 172.21.15.50:0/2459919692 >> 172.21.15.50:0/2459919692 conn(0x7f9f240cbc00 msgr2=0x7f9f240cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:29.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 -- 172.21.15.50:0/2459919692 shutdown_connections 2023-10-19T10:38:29.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.354+0000 7f9f2c241640 1 -- 172.21.15.50:0/2459919692 wait complete. 2023-10-19T10:38:29.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.355+0000 7f9f2c241640 1 Processor -- start 2023-10-19T10:38:29.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 -- start start 2023-10-19T10:38:29.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f24069a70 0x7f9f241f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 0x7f9f240ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f240cb6f0 0x7f9f240c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f241f5f70 con 0x7f9f24069e50 2023-10-19T10:38:29.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f241f34d0 con 0x7f9f240cb6f0 2023-10-19T10:38:29.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f2c241640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f241f2f50 con 0x7f9f24069a70 2023-10-19T10:38:29.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f297dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 0x7f9f240ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f297dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 0x7f9f240ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41894/0 (socket says 172.21.15.50:41894) 2023-10-19T10:38:29.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.356+0000 7f9f297dc640 1 -- 172.21.15.50:0/912993253 learned_addr learned my addr 172.21.15.50:0/912993253 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:29.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f29fdd640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f24069a70 0x7f9f241f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f2a7de640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f240cb6f0 0x7f9f240c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f297dc640 1 -- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f24069a70 msgr2=0x7f9f241f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f297dc640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f24069a70 0x7f9f241f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f297dc640 1 -- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f240cb6f0 msgr2=0x7f9f240c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f297dc640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f240cb6f0 0x7f9f240c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f297dc640 1 -- 172.21.15.50:0/912993253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180094d0 con 0x7f9f24069e50 2023-10-19T10:38:29.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f29fdd640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f24069a70 0x7f9f241f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:29.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f2a7de640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f240cb6f0 0x7f9f240c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:29.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f297dc640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 0x7f9f240ac920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f180090b0 tx=0x7f9f180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004030 con 0x7f9f24069e50 2023-10-19T10:38:29.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18004190 con 0x7f9f24069e50 2023-10-19T10:38:29.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18003610 con 0x7f9f24069e50 2023-10-19T10:38:29.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f241f31f0 con 0x7f9f24069e50 2023-10-19T10:38:29.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.357+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f241f6940 con 0x7f9f24069e50 2023-10-19T10:38:29.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.358+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2404e420 con 0x7f9f24069e50 2023-10-19T10:38:29.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.359+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f18043070 con 0x7f9f24069e50 2023-10-19T10:38:29.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.359+0000 7f9f227fc640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f1006a050 0x7f9f1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.359+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9f180a9fc0 con 0x7f9f24069e50 2023-10-19T10:38:29.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.359+0000 7f9f29fdd640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f1006a050 0x7f9f1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.360+0000 7f9f29fdd640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f1006a050 0x7f9f1006c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009af0 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.363+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9f180dc940 con 0x7f9f24069e50 2023-10-19T10:38:29.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.575+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9f240710f0 con 0x7f9f24069e50 2023-10-19T10:38:29.584 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.582+0000 7f9f227fc640 1 -- 172.21.15.50:0/912993253 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9f18075570 con 0x7f9f24069e50 2023-10-19T10:38:29.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f1006a050 msgr2=0x7f9f1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f1006a050 0x7f9f1006c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009af0 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2023-10-19T10:38:29.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 msgr2=0x7f9f240ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 0x7f9f240ac920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f180090b0 tx=0x7f9f180046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 shutdown_connections 2023-10-19T10:38:29.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9f1006a050 0x7f9f1006c510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f240cb6f0 0x7f9f240c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f24069e50 0x7f9f240ac920 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 --2- 172.21.15.50:0/912993253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f24069a70 0x7f9f241f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 >> 172.21.15.50:0/912993253 conn(0x7f9f240cbc00 msgr2=0x7f9f2415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:29.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 shutdown_connections 2023-10-19T10:38:29.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.584+0000 7f9f2c241640 1 -- 172.21.15.50:0/912993253 wait complete. 2023-10-19T10:38:29.598 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-19T10:38:29.598 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-19T10:38:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.704+0000 7fcd7fa58640 1 Processor -- start 2023-10-19T10:38:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.704+0000 7fcd7fa58640 1 -- start start 2023-10-19T10:38:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7fa58640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcd780b38f0 0x7fcd780d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7fa58640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 0x7fcd78160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7fa58640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd780af250 con 0x7fcd780b3cd0 2023-10-19T10:38:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7fa58640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780b0190 con 0x7fcd780b38f0 2023-10-19T10:38:29.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7fa58640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780af510 con 0x7fcd780b3090 2023-10-19T10:38:29.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7dff5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fcd780b3cd0 0x7fcd780b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58876/0 (socket says 172.21.15.50:58876) 2023-10-19T10:38:29.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.705+0000 7fcd7dff5640 1 -- 172.21.15.50:0/1267390998 learned_addr learned my addr 172.21.15.50:0/1267390998 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:29.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7cff3640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 0x7fcd78160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/1267390998 >> v2:172.21.15.183:3300/0 conn(0x7fcd780b38f0 0x7fcd780d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289407180 0 0) 0x7fcd780af250 con 0x7fcd780b3cd0 2023-10-19T10:38:29.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd64003520 con 0x7fcd780b3cd0 2023-10-19T10:38:29.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576040056 0 0) 0x7fcd64003520 con 0x7fcd780b3cd0 2023-10-19T10:38:29.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 msgr2=0x7fcd78160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 0x7fcd78160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 >> v2:172.21.15.183:3300/0 conn(0x7fcd780b38f0 msgr2=0x7fcd780d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 --2- 172.21.15.50:0/1267390998 >> v2:172.21.15.183:3300/0 conn(0x7fcd780b38f0 0x7fcd780d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd780af7f0 con 0x7fcd780b3cd0 2023-10-19T10:38:29.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fcd700019f0 con 0x7fcd780b3cd0 2023-10-19T10:38:29.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/1267390998 >> v2:172.21.15.183:3300/0 conn(0x7fcd780b38f0 0x7fcd780d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:29.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 >> v1:172.21.15.50:6789/0 conn(0x7fcd780b3cd0 legacy=0x7fcd780b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64003970 0x7fcd64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd640047c0 0x7fcd64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 0x7fcd64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780b0190 con 0x7fcd640047c0 2023-10-19T10:38:29.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780af510 con 0x7fcd64003970 2023-10-19T10:38:29.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6400b980 con 0x7fcd780b3090 2023-10-19T10:38:29.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7cff3640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd640047c0 0x7fcd64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64003970 0x7fcd64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7dff5640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 0x7fcd64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7cff3640 1 -- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 msgr2=0x7fcd64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.706+0000 7fcd7cff3640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd780b3090 0x7fcd64005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd7cff3640 1 -- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64003970 msgr2=0x7fcd64003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:29.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd7cff3640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64003970 0x7fcd64003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd7cff3640 1 -- 172.21.15.50:0/1267390998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64003520 con 0x7fcd640047c0 2023-10-19T10:38:29.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd7cff3640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd640047c0 0x7fcd64004c40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009a90 tx=0x7fcd640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beff6cd5ea89c8f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6802c680 con 0x7fcd640047c0 2023-10-19T10:38:29.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6802c7e0 con 0x7fcd640047c0 2023-10-19T10:38:29.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.707+0000 7fcd6ffff640 1 -- 172.21.15.50:0/1267390998 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6802cb70 con 0x7fcd640047c0 2023-10-19T10:38:29.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7fa58640 1 -- 172.21.15.50:0/1267390998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd640047c0 msgr2=0x7fcd64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/1267390998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd640047c0 0x7fcd64004c40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009a90 tx=0x7fcd640057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7dff5640 1 -- 172.21.15.50:0/1267390998 reap_dead start 2023-10-19T10:38:29.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7fa58640 1 -- 172.21.15.50:0/1267390998 shutdown_connections 2023-10-19T10:38:29.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7fa58640 1 -- 172.21.15.50:0/1267390998 >> 172.21.15.50:0/1267390998 conn(0x7fcd7809b960 msgr2=0x7fcd780d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:29.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7fa58640 1 -- 172.21.15.50:0/1267390998 shutdown_connections 2023-10-19T10:38:29.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.708+0000 7fcd7fa58640 1 -- 172.21.15.50:0/1267390998 wait complete. 2023-10-19T10:38:29.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.709+0000 7fcd7fa58640 1 Processor -- start 2023-10-19T10:38:29.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.709+0000 7fcd7fa58640 1 -- start start 2023-10-19T10:38:29.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7fa58640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd640047c0 0x7fcd7814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7fa58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd7815b3d0 0x7fcd78148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7fa58640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 0x7fcd780c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7fa58640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815c980 con 0x7fcd7815b830 2023-10-19T10:38:29.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7fa58640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815bd00 con 0x7fcd7815b3d0 2023-10-19T10:38:29.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7fa58640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815bfe0 con 0x7fcd640047c0 2023-10-19T10:38:29.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 0x7fcd780c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 0x7fcd780c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41916/0 (socket says 172.21.15.50:41916) 2023-10-19T10:38:29.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 -- 172.21.15.50:0/854354281 learned_addr learned my addr 172.21.15.50:0/854354281 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:29.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd640047c0 0x7fcd7814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7cff3640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd7815b3d0 0x7fcd78148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 -- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd640047c0 msgr2=0x7fcd7814bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd640047c0 0x7fcd7814bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 -- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd7815b3d0 msgr2=0x7fcd78148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:29.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd7815b3d0 0x7fcd78148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:29.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7dff5640 1 -- 172.21.15.50:0/854354281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7815c2c0 con 0x7fcd7815b830 2023-10-19T10:38:29.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.710+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd640047c0 0x7fcd7814bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:29.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.711+0000 7fcd7dff5640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 0x7fcd780c7240 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd70000f80 tx=0x7fcd70007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.711+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000ac80 con 0x7fcd7815b830 2023-10-19T10:38:29.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.711+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000ae20 con 0x7fcd7815b830 2023-10-19T10:38:29.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.711+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7815c5a0 con 0x7fcd7815b830 2023-10-19T10:38:29.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.711+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70013630 con 0x7fcd7815b830 2023-10-19T10:38:29.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.711+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd78148b70 con 0x7fcd7815b830 2023-10-19T10:38:29.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.712+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd780afd90 con 0x7fcd7815b830 2023-10-19T10:38:29.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.712+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcd70013790 con 0x7fcd7815b830 2023-10-19T10:38:29.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.713+0000 7fcd6dffb640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcd5c06a050 0x7fcd5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:29.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.713+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcd70088e50 con 0x7fcd7815b830 2023-10-19T10:38:29.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.713+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcd5c06a050 0x7fcd5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:29.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.714+0000 7fcd7d7f4640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcd5c06a050 0x7fcd5c06c510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009dc0 tx=0x7fcd74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:29.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.717+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fcd780afd90 con 0x7fcd7815b830 2023-10-19T10:38:29.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:29.931+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcd780afd90 con 0x7fcd7815b830 2023-10-19T10:38:30.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.592+0000 7fcd6dffb640 1 -- 172.21.15.50:0/854354281 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd780afd90 con 0x7fcd7815b830 2023-10-19T10:38:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcd5c06a050 msgr2=0x7fcd5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcd5c06a050 0x7fcd5c06c510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009dc0 tx=0x7fcd74008040 comp rx=0 tx=0).stop 2023-10-19T10:38:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 msgr2=0x7fcd780c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 0x7fcd780c7240 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd70000f80 tx=0x7fcd70007b20 comp rx=0 tx=0).stop 2023-10-19T10:38:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 shutdown_connections 2023-10-19T10:38:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcd5c06a050 0x7fcd5c06c510 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd7815b830 0x7fcd780c7240 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd7815b3d0 0x7fcd78148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 --2- 172.21.15.50:0/854354281 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd640047c0 0x7fcd7814bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 >> 172.21.15.50:0/854354281 conn(0x7fcd7809b960 msgr2=0x7fcd780c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:30.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 shutdown_connections 2023-10-19T10:38:30.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.595+0000 7fcd7fa58640 1 -- 172.21.15.50:0/854354281 wait complete. 2023-10-19T10:38:30.611 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-19T10:38:30.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:30.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 Processor -- start 2023-10-19T10:38:30.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 -- start start 2023-10-19T10:38:30.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f94b40b2400 0x7f94b415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:30.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f94b4160100 0x7f94b4160510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:30.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b41610a0 con 0x7f94b40ce3c0 2023-10-19T10:38:30.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b415cad0 con 0x7f94b4160100 2023-10-19T10:38:30.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94bb22f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b40ce7a0 con 0x7f94b40b2400 2023-10-19T10:38:30.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.723+0000 7f94b97cc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f94b4160100 0x7f94b4160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:30.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b97cc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f94b4160100 0x7f94b4160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41918/0 (socket says 172.21.15.50:41918) 2023-10-19T10:38:30.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b97cc640 1 -- 172.21.15.50:0/3395268391 learned_addr learned my addr 172.21.15.50:0/3395268391 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:30.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b3fff640 1 --2- 172.21.15.50:0/3395268391 >> v2:172.21.15.183:3300/0 conn(0x7f94b40b2400 0x7f94b415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:30.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b97cc640 1 -- 172.21.15.50:0/3395268391 >> v2:172.21.15.183:3300/0 conn(0x7f94b40b2400 msgr2=0x7f94b415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:30.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b97cc640 1 --2- 172.21.15.50:0/3395268391 >> v2:172.21.15.183:3300/0 conn(0x7f94b40b2400 0x7f94b415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b97cc640 1 -- 172.21.15.50:0/3395268391 >> v1:172.21.15.183:6789/0 conn(0x7f94b40ce3c0 legacy=0x7f94b40b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.724+0000 7f94b97cc640 1 -- 172.21.15.50:0/3395268391 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4160dc0 con 0x7f94b4160100 2023-10-19T10:38:30.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94b97cc640 1 --2- 172.21.15.50:0/3395268391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b4160510 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f94b4145cb0 tx=0x7f949c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b67497dc6a917b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:30.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94b37fe640 1 -- 172.21.15.50:0/3395268391 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c030860 con 0x7f94b4160100 2023-10-19T10:38:30.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94b37fe640 1 -- 172.21.15.50:0/3395268391 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c0309c0 con 0x7f94b4160100 2023-10-19T10:38:30.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94b37fe640 1 -- 172.21.15.50:0/3395268391 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c030d50 con 0x7f94b4160100 2023-10-19T10:38:30.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 -- 172.21.15.50:0/3395268391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 msgr2=0x7f94b4160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 --2- 172.21.15.50:0/3395268391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b4160510 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f94b4145cb0 tx=0x7f949c02b9a0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 -- 172.21.15.50:0/3395268391 shutdown_connections 2023-10-19T10:38:30.734 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 --2- 172.21.15.50:0/3395268391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b4160510 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 --2- 172.21.15.50:0/3395268391 >> v2:172.21.15.183:3300/0 conn(0x7f94b40b2400 0x7f94b415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 -- 172.21.15.50:0/3395268391 >> 172.21.15.50:0/3395268391 conn(0x7f94b406f3f0 msgr2=0x7f94b406f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:30.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 -- 172.21.15.50:0/3395268391 shutdown_connections 2023-10-19T10:38:30.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.725+0000 7f94bb22f640 1 -- 172.21.15.50:0/3395268391 wait complete. 2023-10-19T10:38:30.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.727+0000 7f94bb22f640 1 Processor -- start 2023-10-19T10:38:30.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.727+0000 7f94bb22f640 1 -- start start 2023-10-19T10:38:30.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b40b2400 0x7f94b41f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:30.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94b40ce3c0 0x7f94b40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:30.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b40b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:30.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41f1dd0 con 0x7f94b4160100 2023-10-19T10:38:30.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40c8900 con 0x7f94b40b2400 2023-10-19T10:38:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b415e910 con 0x7f94b40ce3c0 2023-10-19T10:38:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b40b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b40b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41926/0 (socket says 172.21.15.50:41926) 2023-10-19T10:38:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 -- 172.21.15.50:0/2547102925 learned_addr learned my addr 172.21.15.50:0/2547102925 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:30.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b8fcb640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b40b2400 0x7f94b41f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:30.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b3fff640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94b40ce3c0 0x7f94b40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:30.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 -- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94b40ce3c0 msgr2=0x7f94b40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94b40ce3c0 0x7f94b40b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 -- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b40b2400 msgr2=0x7f94b41f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b40b2400 0x7f94b41f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 -- 172.21.15.50:0/2547102925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0094d0 con 0x7f94b4160100 2023-10-19T10:38:30.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b97cc640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b40b5970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f949c0090b0 tx=0x7f949c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:30.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c039a90 con 0x7f94b4160100 2023-10-19T10:38:30.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c039bf0 con 0x7f94b4160100 2023-10-19T10:38:30.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.728+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b415ebf0 con 0x7f94b4160100 2023-10-19T10:38:30.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.729+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c043410 con 0x7f94b4160100 2023-10-19T10:38:30.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.729+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b415f1b0 con 0x7f94b4160100 2023-10-19T10:38:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.729+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4073990 con 0x7f94b4160100 2023-10-19T10:38:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.730+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f949c04c3b0 con 0x7f94b4160100 2023-10-19T10:38:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.730+0000 7f94b17fa640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f949406a050 0x7f949406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.730+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f949c0aaad0 con 0x7f94b4160100 2023-10-19T10:38:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.731+0000 7f94b8fcb640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f949406a050 0x7f949406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.731+0000 7f94b8fcb640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f949406a050 0x7f949406c510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0012e0 tx=0x7f94ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.735+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f949c0761c0 con 0x7f94b4160100 2023-10-19T10:38:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.944+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94b40cd590 con 0x7f94b4160100 2023-10-19T10:38:30.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.945+0000 7f94b17fa640 1 -- 172.21.15.50:0/2547102925 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f949c0797e0 con 0x7f94b4160100 2023-10-19T10:38:30.948 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:30.948 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:38:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:38:30.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f949406a050 msgr2=0x7f949406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f949406a050 0x7f949406c510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0012e0 tx=0x7f94ac009040 comp rx=0 tx=0).stop 2023-10-19T10:38:30.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 msgr2=0x7f94b40b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:30.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b40b5970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f949c0090b0 tx=0x7f949c004730 comp rx=0 tx=0).stop 2023-10-19T10:38:30.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 shutdown_connections 2023-10-19T10:38:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f949406a050 0x7f949406c510 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94b4160100 0x7f94b40b5970 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94b40ce3c0 0x7f94b40b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 --2- 172.21.15.50:0/2547102925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b40b2400 0x7f94b41f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:30.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 >> 172.21.15.50:0/2547102925 conn(0x7f94b406f3f0 msgr2=0x7f94b406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:30.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 shutdown_connections 2023-10-19T10:38:30.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:30.947+0000 7f94bb22f640 1 -- 172.21.15.50:0/2547102925 wait complete. 2023-10-19T10:38:30.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:38:31.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.064+0000 7f60e01d9640 1 Processor -- start 2023-10-19T10:38:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.064+0000 7f60e01d9640 1 -- start start 2023-10-19T10:38:31.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60e01d9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80d1830 0x7f60d81670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60e01d9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80b6de0 con 0x7f60d815c700 2023-10-19T10:38:31.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60e01d9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80b70c0 con 0x7f60d80cd300 2023-10-19T10:38:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60e01d9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80b7960 con 0x7f60d80d1830 2023-10-19T10:38:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60dd774640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f60d815c700 0x7f60d80b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58890/0 (socket says 172.21.15.50:58890) 2023-10-19T10:38:31.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60dd774640 1 -- 172.21.15.50:0/3703576073 learned_addr learned my addr 172.21.15.50:0/3703576073 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:31.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60de776640 1 --2- 172.21.15.50:0/3703576073 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80d1830 0x7f60d81670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276017805 0 0) 0x7f60d80b6de0 con 0x7f60d815c700 2023-10-19T10:38:31.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60c4003520 con 0x7f60d815c700 2023-10-19T10:38:31.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.065+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261854373 0 0) 0x7f60d80b70c0 con 0x7f60d80cd300 2023-10-19T10:38:31.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.066+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d80b6de0 con 0x7f60d80cd300 2023-10-19T10:38:31.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.066+0000 7f60de776640 1 -- 172.21.15.50:0/3703576073 >> v1:172.21.15.183:6789/0 conn(0x7f60d80cd300 legacy=0x7f60d815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.066+0000 7f60de776640 1 -- 172.21.15.50:0/3703576073 >> v1:172.21.15.50:6789/0 conn(0x7f60d815c700 legacy=0x7f60d80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.066+0000 7f60de776640 1 -- 172.21.15.50:0/3703576073 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80b73a0 con 0x7f60d80d1830 2023-10-19T10:38:31.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.066+0000 7f60de776640 1 --2- 172.21.15.50:0/3703576073 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80d1830 0x7f60d81670d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f60d00097d0 tx=0x7f60d002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3cbf11b55737dc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0031890 con 0x7f60d80d1830 2023-10-19T10:38:31.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d00319f0 con 0x7f60d80d1830 2023-10-19T10:38:31.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60dcf73640 1 -- 172.21.15.50:0/3703576073 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0031d60 con 0x7f60d80d1830 2023-10-19T10:38:31.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 -- 172.21.15.50:0/3703576073 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80d1830 msgr2=0x7f60d81670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 --2- 172.21.15.50:0/3703576073 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80d1830 0x7f60d81670d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f60d00097d0 tx=0x7f60d002b930 comp rx=0 tx=0).stop 2023-10-19T10:38:31.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 -- 172.21.15.50:0/3703576073 shutdown_connections 2023-10-19T10:38:31.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 --2- 172.21.15.50:0/3703576073 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80d1830 0x7f60d81670d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 -- 172.21.15.50:0/3703576073 >> 172.21.15.50:0/3703576073 conn(0x7f60d8071320 msgr2=0x7f60d8071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:31.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 -- 172.21.15.50:0/3703576073 shutdown_connections 2023-10-19T10:38:31.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.067+0000 7f60e01d9640 1 -- 172.21.15.50:0/3703576073 wait complete. 2023-10-19T10:38:31.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.068+0000 7f60e01d9640 1 Processor -- start 2023-10-19T10:38:31.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.068+0000 7f60e01d9640 1 -- start start 2023-10-19T10:38:31.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60e01d9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80cd300 0x7f60d814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60e01d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 0x7f60d80c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60e01d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d815c700 0x7f60d80c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60e01d9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80c8dd0 con 0x7f60d80d1830 2023-10-19T10:38:31.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60e01d9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d815dfe0 con 0x7f60d815c700 2023-10-19T10:38:31.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60e01d9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d815f9e0 con 0x7f60d80cd300 2023-10-19T10:38:31.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 0x7f60d80c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 0x7f60d80c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41928/0 (socket says 172.21.15.50:41928) 2023-10-19T10:38:31.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 -- 172.21.15.50:0/1291329935 learned_addr learned my addr 172.21.15.50:0/1291329935 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:31.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60de776640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d815c700 0x7f60d80c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60ddf75640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80cd300 0x7f60d814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 -- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80cd300 msgr2=0x7f60d814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80cd300 0x7f60d814a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 -- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d815c700 msgr2=0x7f60d80c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d815c700 0x7f60d80c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.069+0000 7f60dd774640 1 -- 172.21.15.50:0/1291329935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d0009400 con 0x7f60d80d1830 2023-10-19T10:38:31.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.070+0000 7f60dd774640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 0x7f60d80c8890 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c430 tx=0x7f60c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.070+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8005ce0 con 0x7f60d80d1830 2023-10-19T10:38:31.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.070+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c8005e40 con 0x7f60d80d1830 2023-10-19T10:38:31.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.070+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d815d6b0 con 0x7f60d80d1830 2023-10-19T10:38:31.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.070+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c80072f0 con 0x7f60d80d1830 2023-10-19T10:38:31.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.070+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d815dc70 con 0x7f60d80d1830 2023-10-19T10:38:31.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.071+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8073990 con 0x7f60d80d1830 2023-10-19T10:38:31.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.071+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f60c8007450 con 0x7f60d80d1830 2023-10-19T10:38:31.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.072+0000 7f60ce7fc640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f60bc06a050 0x7f60bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.072+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f60c8087cb0 con 0x7f60d80d1830 2023-10-19T10:38:31.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.072+0000 7f60ddf75640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f60bc06a050 0x7f60bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.072+0000 7f60ddf75640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f60bc06a050 0x7f60bc06c510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60d4001cc0 tx=0x7f60d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.076+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f60c80532f0 con 0x7f60d80d1830 2023-10-19T10:38:31.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.252+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60d80d2f10 con 0x7f60d80d1830 2023-10-19T10:38:31.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.253+0000 7f60ce7fc640 1 -- 172.21.15.50:0/1291329935 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18464 (secure 0 0 0) 0x7f60c8056910 con 0x7f60d80d1830 2023-10-19T10:38:31.256 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:31.256 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":42,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:38:29.580320+0000","last_up_change":"2023-10-19T10:36:42.521118+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6806","nonce":415608244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6809","nonce":415608244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6818","nonce":415608244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":415608244},{"type":"v1","addr":"172.21.15.50:6814","nonce":415608244}]},"public_addr":"172.21.15.50:6806/415608244","cluster_addr":"172.21.15.50:6809/415608244","heartbeat_back_addr":"172.21.15.50:6818/415608244","heartbeat_front_addr":"172.21.15.50:6814/415608244","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:38:31.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f60bc06a050 msgr2=0x7f60bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f60bc06a050 0x7f60bc06c510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60d4001cc0 tx=0x7f60d4009040 comp rx=0 tx=0).stop 2023-10-19T10:38:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 msgr2=0x7f60d80c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 0x7f60d80c8890 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c430 tx=0x7f60c800c8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 shutdown_connections 2023-10-19T10:38:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f60bc06a050 0x7f60bc06c510 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d815c700 0x7f60d80c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60d80d1830 0x7f60d80c8890 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 --2- 172.21.15.50:0/1291329935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60d80cd300 0x7f60d814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 >> 172.21.15.50:0/1291329935 conn(0x7f60d8071320 msgr2=0x7f60d8071700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:31.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 shutdown_connections 2023-10-19T10:38:31.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.255+0000 7f60e01d9640 1 -- 172.21.15.50:0/1291329935 wait complete. 2023-10-19T10:38:31.268 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-19T10:38:31.269 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:38:31.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.372+0000 7f37b2478640 1 Processor -- start 2023-10-19T10:38:31.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.372+0000 7f37b2478640 1 -- start start 2023-10-19T10:38:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b2478640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b2478640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37ac0cd920 0x7f37ac0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b2478640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06aa30 con 0x7f37ac15c570 2023-10-19T10:38:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b2478640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac15ce90 con 0x7f37ac0cd920 2023-10-19T10:38:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b2478640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac06b2d0 con 0x7f37ac0d0e30 2023-10-19T10:38:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b0a15640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37ac0cd920 0x7f37ac0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b0a15640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f37ac0cd920 0x7f37ac0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41938/0 (socket says 172.21.15.50:41938) 2023-10-19T10:38:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.373+0000 7f37b0a15640 1 -- 172.21.15.50:0/1380790040 learned_addr learned my addr 172.21.15.50:0/1380790040 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37ab7fe640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37b0a15640 1 -- 172.21.15.50:0/1380790040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 msgr2=0x7f37ac15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37b0a15640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37b0a15640 1 -- 172.21.15.50:0/1380790040 >> v1:172.21.15.183:6789/0 conn(0x7f37ac15c570 legacy=0x7f37ac06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37b0a15640 1 -- 172.21.15.50:0/1380790040 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0d1270 con 0x7f37ac0cd920 2023-10-19T10:38:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37ab7fe640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:31.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.374+0000 7f37b0a15640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac0d26a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009e50 tx=0x7f37a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b24fada850f12a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37aaffd640 1 -- 172.21.15.50:0/1380790040 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a402ca70 con 0x7f37ac0cd920 2023-10-19T10:38:31.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37aaffd640 1 -- 172.21.15.50:0/1380790040 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a402cbd0 con 0x7f37ac0cd920 2023-10-19T10:38:31.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37aaffd640 1 -- 172.21.15.50:0/1380790040 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40343f0 con 0x7f37ac0cd920 2023-10-19T10:38:31.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 -- 172.21.15.50:0/1380790040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 msgr2=0x7f37ac0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac0d26a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009e50 tx=0x7f37a402bed0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 -- 172.21.15.50:0/1380790040 shutdown_connections 2023-10-19T10:38:31.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac0d26a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 --2- 172.21.15.50:0/1380790040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 -- 172.21.15.50:0/1380790040 >> 172.21.15.50:0/1380790040 conn(0x7f37ac089ff0 msgr2=0x7f37ac09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:31.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 -- 172.21.15.50:0/1380790040 shutdown_connections 2023-10-19T10:38:31.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.375+0000 7f37b2478640 1 -- 172.21.15.50:0/1380790040 wait complete. 2023-10-19T10:38:31.384 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.376+0000 7f37b2478640 1 Processor -- start 2023-10-19T10:38:31.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 -- start start 2023-10-19T10:38:31.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac1f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ac15c570 0x7f37ac0cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac15a830 con 0x7f37ac0cd920 2023-10-19T10:38:31.385 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0b6510 con 0x7f37ac15c570 2023-10-19T10:38:31.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37b2478640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1f20e0 con 0x7f37ac0d0e30 2023-10-19T10:38:31.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37abfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37abfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41948/0 (socket says 172.21.15.50:41948) 2023-10-19T10:38:31.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.377+0000 7f37abfff640 1 -- 172.21.15.50:0/504709401 learned_addr learned my addr 172.21.15.50:0/504709401 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:31.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37b0a15640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ac15c570 0x7f37ac0cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37ab7fe640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac1f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37abfff640 1 -- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 msgr2=0x7f37ac1f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37abfff640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac1f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37abfff640 1 -- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ac15c570 msgr2=0x7f37ac0cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37abfff640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ac15c570 0x7f37ac0cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37abfff640 1 -- 172.21.15.50:0/504709401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009b30 con 0x7f37ac0cd920 2023-10-19T10:38:31.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37abfff640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac15a2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c450 tx=0x7f379c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0052d0 con 0x7f37ac0cd920 2023-10-19T10:38:31.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c005450 con 0x7f37ac0cd920 2023-10-19T10:38:31.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c003880 con 0x7f37ac0cd920 2023-10-19T10:38:31.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac1f23c0 con 0x7f37ac0cd920 2023-10-19T10:38:31.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.378+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac0b67b0 con 0x7f37ac0cd920 2023-10-19T10:38:31.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.379+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac073990 con 0x7f37ac0cd920 2023-10-19T10:38:31.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.380+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f379c0039e0 con 0x7f37ac0cd920 2023-10-19T10:38:31.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.380+0000 7f37a8ff9640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f379806a050 0x7f379806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.380+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f379c088290 con 0x7f37ac0cd920 2023-10-19T10:38:31.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.380+0000 7f37ab7fe640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f379806a050 0x7f379806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.381+0000 7f37ab7fe640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f379806a050 0x7f379806c510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0cc130 tx=0x7f37a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.384+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f379c053820 con 0x7f37ac0cd920 2023-10-19T10:38:31.551 INFO:tasks.ceph.mds.a.smithi050.stderr:2023-10-19T10:38:31.548+0000 7f9899a15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-19T10:38:31.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.556+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37ac06ac80 con 0x7f379806a050 2023-10-19T10:38:31.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.565+0000 7f37a8ff9640 1 -- 172.21.15.50:0/504709401 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+298251 (secure 0 0 0) 0x7f37a000b6b0 con 0x7f379806a050 2023-10-19T10:38:31.570 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:31.572 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:38:31.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f379806a050 msgr2=0x7f379806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f379806a050 0x7f379806c510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0cc130 tx=0x7f37a0008040 comp rx=0 tx=0).stop 2023-10-19T10:38:31.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 msgr2=0x7f37ac15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac15a2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c450 tx=0x7f379c00c910 comp rx=0 tx=0).stop 2023-10-19T10:38:31.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 shutdown_connections 2023-10-19T10:38:31.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f379806a050 0x7f379806c510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ac15c570 0x7f37ac0cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37ac0d0e30 0x7f37ac1f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 --2- 172.21.15.50:0/504709401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37ac0cd920 0x7f37ac15a2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 >> 172.21.15.50:0/504709401 conn(0x7f37ac089ff0 msgr2=0x7f37ac0cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:31.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 shutdown_connections 2023-10-19T10:38:31.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.568+0000 7f37b2478640 1 -- 172.21.15.50:0/504709401 wait complete. 2023-10-19T10:38:31.581 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-10-19T10:38:30.489504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237156,"kb_used_data":4608,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737564,"statfs":{"total":773094113280,"available":772851265536,"internally_reserved":0,"allocated":4718592,"data_stored":2719326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576210+0000","last_change":"2023-10-19T10:38:27.576210+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576210+0000","last_undegraded":"2023-10-19T10:38:27.576210+0000","last_fullsized":"2023-10-19T10:38:27.576210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576153+0000","last_change":"2023-10-19T10:38:27.576153+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576153+0000","last_undegraded":"2023-10-19T10:38:27.576153+0000","last_fullsized":"2023-10-19T10:38:27.576153+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589712+0000","last_change":"2023-10-19T10:38:27.584383+0000","last_active":"2023-10-19T10:38:27.589712+0000","last_peered":"2023-10-19T10:38:27.589712+0000","last_clean":"2023-10-19T10:38:27.589712+0000","last_became_active":"2023-10-19T10:38:27.583945+0000","last_became_peered":"2023-10-19T10:38:27.583945+0000","last_unstale":"2023-10-19T10:38:27.589712+0000","last_undegraded":"2023-10-19T10:38:27.589712+0000","last_fullsized":"2023-10-19T10:38:27.589712+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:17:13.906630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575423+0000","last_change":"2023-10-19T10:38:27.575423+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575423+0000","last_undegraded":"2023-10-19T10:38:27.575423+0000","last_fullsized":"2023-10-19T10:38:27.575423+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.592015+0000","last_change":"2023-10-19T10:38:27.589861+0000","last_active":"2023-10-19T10:38:27.592015+0000","last_peered":"2023-10-19T10:38:27.592015+0000","last_clean":"2023-10-19T10:38:27.592015+0000","last_became_active":"2023-10-19T10:38:27.589594+0000","last_became_peered":"2023-10-19T10:38:27.589594+0000","last_unstale":"2023-10-19T10:38:27.592015+0000","last_undegraded":"2023-10-19T10:38:27.592015+0000","last_fullsized":"2023-10-19T10:38:27.592015+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:32:26.735762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575835+0000","last_change":"2023-10-19T10:38:27.575835+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575835+0000","last_undegraded":"2023-10-19T10:38:27.575835+0000","last_fullsized":"2023-10-19T10:38:27.575835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589683+0000","last_change":"2023-10-19T10:38:27.584475+0000","last_active":"2023-10-19T10:38:27.589683+0000","last_peered":"2023-10-19T10:38:27.589683+0000","last_clean":"2023-10-19T10:38:27.589683+0000","last_became_active":"2023-10-19T10:38:27.584175+0000","last_became_peered":"2023-10-19T10:38:27.584175+0000","last_unstale":"2023-10-19T10:38:27.589683+0000","last_undegraded":"2023-10-19T10:38:27.589683+0000","last_fullsized":"2023-10-19T10:38:27.589683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:45:33.090392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575636+0000","last_change":"2023-10-19T10:38:27.575636+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575636+0000","last_undegraded":"2023-10-19T10:38:27.575636+0000","last_fullsized":"2023-10-19T10:38:27.575636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.588906+0000","last_change":"2023-10-19T10:38:27.584351+0000","last_active":"2023-10-19T10:38:27.588906+0000","last_peered":"2023-10-19T10:38:27.588906+0000","last_clean":"2023-10-19T10:38:27.588906+0000","last_became_active":"2023-10-19T10:38:27.583963+0000","last_became_peered":"2023-10-19T10:38:27.583963+0000","last_unstale":"2023-10-19T10:38:27.588906+0000","last_undegraded":"2023-10-19T10:38:27.588906+0000","last_fullsized":"2023-10-19T10:38:27.588906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:16.906120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594279+0000","last_change":"2023-10-19T10:38:27.590766+0000","last_active":"2023-10-19T10:38:27.594279+0000","last_peered":"2023-10-19T10:38:27.594279+0000","last_clean":"2023-10-19T10:38:27.594279+0000","last_became_active":"2023-10-19T10:38:27.590488+0000","last_became_peered":"2023-10-19T10:38:27.590488+0000","last_unstale":"2023-10-19T10:38:27.594279+0000","last_undegraded":"2023-10-19T10:38:27.594279+0000","last_fullsized":"2023-10-19T10:38:27.594279+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:19:14.891257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.124735+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:38:22.124735+0000","last_peered":"2023-10-19T10:38:22.124735+0000","last_clean":"2023-10-19T10:38:22.124735+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:38:22.124735+0000","last_undegraded":"2023-10-19T10:38:22.124735+0000","last_fullsized":"2023-10-19T10:38:22.124735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:53:23.212478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655521+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:38:22.655521+0000","last_peered":"2023-10-19T10:38:22.655521+0000","last_clean":"2023-10-19T10:38:22.655521+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:38:22.655521+0000","last_undegraded":"2023-10-19T10:38:22.655521+0000","last_fullsized":"2023-10-19T10:38:22.655521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:33:52.868361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.578327+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:38:27.578327+0000","last_peered":"2023-10-19T10:38:27.578327+0000","last_clean":"2023-10-19T10:38:27.578327+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:38:27.578327+0000","last_undegraded":"2023-10-19T10:38:27.578327+0000","last_fullsized":"2023-10-19T10:38:27.578327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:15.047277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655464+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:38:22.655464+0000","last_peered":"2023-10-19T10:38:22.655464+0000","last_clean":"2023-10-19T10:38:22.655464+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:38:22.655464+0000","last_undegraded":"2023-10-19T10:38:22.655464+0000","last_fullsized":"2023-10-19T10:38:22.655464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:50:47.137818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655692+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:38:22.655692+0000","last_peered":"2023-10-19T10:38:22.655692+0000","last_clean":"2023-10-19T10:38:22.655692+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:38:22.655692+0000","last_undegraded":"2023-10-19T10:38:22.655692+0000","last_fullsized":"2023-10-19T10:38:22.655692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:06:59.602523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594316+0000","last_change":"2023-10-19T10:38:27.590779+0000","last_active":"2023-10-19T10:38:27.594316+0000","last_peered":"2023-10-19T10:38:27.594316+0000","last_clean":"2023-10-19T10:38:27.594316+0000","last_became_active":"2023-10-19T10:38:27.590482+0000","last_became_peered":"2023-10-19T10:38:27.590482+0000","last_unstale":"2023-10-19T10:38:27.594316+0000","last_undegraded":"2023-10-19T10:38:27.594316+0000","last_fullsized":"2023-10-19T10:38:27.594316+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:10:42.478900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.123623+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:38:22.123623+0000","last_peered":"2023-10-19T10:38:22.123623+0000","last_clean":"2023-10-19T10:38:22.123623+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:38:22.123623+0000","last_undegraded":"2023-10-19T10:38:22.123623+0000","last_fullsized":"2023-10-19T10:38:22.123623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:13:16.360504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575245+0000","last_change":"2023-10-19T10:38:27.575245+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575245+0000","last_undegraded":"2023-10-19T10:38:27.575245+0000","last_fullsized":"2023-10-19T10:38:27.575245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.576832+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:38:27.576832+0000","last_peered":"2023-10-19T10:38:27.576832+0000","last_clean":"2023-10-19T10:38:27.576832+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:38:27.576832+0000","last_undegraded":"2023-10-19T10:38:27.576832+0000","last_fullsized":"2023-10-19T10:38:27.576832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:43:37.587705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655688+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:38:22.655688+0000","last_peered":"2023-10-19T10:38:22.655688+0000","last_clean":"2023-10-19T10:38:22.655688+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:38:22.655688+0000","last_undegraded":"2023-10-19T10:38:22.655688+0000","last_fullsized":"2023-10-19T10:38:22.655688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:47:46.186082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.577404+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:38:27.577404+0000","last_peered":"2023-10-19T10:38:27.577404+0000","last_clean":"2023-10-19T10:38:27.577404+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:38:27.577404+0000","last_undegraded":"2023-10-19T10:38:27.577404+0000","last_fullsized":"2023-10-19T10:38:27.577404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:45.342865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":794443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.501}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.377},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.438}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.441},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.505}]},{"osd":2,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.754},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.752},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":781013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.307}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.374}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.621}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:38:31.588 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:38:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.725+0000 7f3d038b1640 1 Processor -- start 2023-10-19T10:38:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.725+0000 7f3d038b1640 1 -- start start 2023-10-19T10:38:31.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d038b1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3cfc15c570 0x7f3cfc0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d038b1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3cfc15c950 0x7f3cfc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d038b1640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0cde60 con 0x7f3cfc06b760 2023-10-19T10:38:31.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d038b1640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc06aa30 con 0x7f3cfc15c570 2023-10-19T10:38:31.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d038b1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc06b2d0 con 0x7f3cfc15c950 2023-10-19T10:38:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d0164d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3cfc15c570 0x7f3cfc0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d01e4e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3cfc06b760 0x7f3cfc0d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58906/0 (socket says 172.21.15.50:58906) 2023-10-19T10:38:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.726+0000 7f3d01e4e640 1 -- 172.21.15.50:0/1208833859 learned_addr learned my addr 172.21.15.50:0/1208833859 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3d00e4c640 1 --2- 172.21.15.50:0/1208833859 >> v2:172.21.15.183:3300/0 conn(0x7f3cfc15c950 0x7f3cfc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3d0164d640 1 -- 172.21.15.50:0/1208833859 >> v2:172.21.15.183:3300/0 conn(0x7f3cfc15c950 msgr2=0x7f3cfc0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3d0164d640 1 --2- 172.21.15.50:0/1208833859 >> v2:172.21.15.183:3300/0 conn(0x7f3cfc15c950 0x7f3cfc0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3d0164d640 1 -- 172.21.15.50:0/1208833859 >> v1:172.21.15.50:6789/0 conn(0x7f3cfc06b760 legacy=0x7f3cfc0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3d0164d640 1 -- 172.21.15.50:0/1208833859 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0ce090 con 0x7f3cfc15c570 2023-10-19T10:38:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3cfbfff640 1 -- 172.21.15.50:0/1208833859 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121041858 0 0) 0x7f3cfc0cde60 con 0x7f3cfc06b760 2023-10-19T10:38:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.727+0000 7f3d0164d640 1 --2- 172.21.15.50:0/1208833859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0d0e30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0b87c0 tx=0x7f3cf402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c7092079370e98 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3cfbfff640 1 -- 172.21.15.50:0/1208833859 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf402e070 con 0x7f3cfc15c570 2023-10-19T10:38:31.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3cfbfff640 1 -- 172.21.15.50:0/1208833859 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf40315a0 con 0x7f3cfc15c570 2023-10-19T10:38:31.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3cfbfff640 1 -- 172.21.15.50:0/1208833859 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4031930 con 0x7f3cfc15c570 2023-10-19T10:38:31.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 -- 172.21.15.50:0/1208833859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc15c570 msgr2=0x7f3cfc0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 --2- 172.21.15.50:0/1208833859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0d0e30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0b87c0 tx=0x7f3cf402b840 comp rx=0 tx=0).stop 2023-10-19T10:38:31.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 -- 172.21.15.50:0/1208833859 shutdown_connections 2023-10-19T10:38:31.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 --2- 172.21.15.50:0/1208833859 >> v2:172.21.15.183:3300/0 conn(0x7f3cfc15c950 0x7f3cfc0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 --2- 172.21.15.50:0/1208833859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0d0e30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 -- 172.21.15.50:0/1208833859 >> 172.21.15.50:0/1208833859 conn(0x7f3cfc089ff0 msgr2=0x7f3cfc09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:31.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 -- 172.21.15.50:0/1208833859 shutdown_connections 2023-10-19T10:38:31.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.728+0000 7f3d038b1640 1 -- 172.21.15.50:0/1208833859 wait complete. 2023-10-19T10:38:31.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.729+0000 7f3d038b1640 1 Processor -- start 2023-10-19T10:38:31.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.729+0000 7f3d038b1640 1 -- start start 2023-10-19T10:38:31.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d038b1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 0x7f3cfc144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d038b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d038b1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3cfc15c950 0x7f3cfc0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d038b1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0c97e0 con 0x7f3cfc06b760 2023-10-19T10:38:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d038b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc156bb0 con 0x7f3cfc15c570 2023-10-19T10:38:31.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d038b1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc156e90 con 0x7f3cfc15c950 2023-10-19T10:38:31.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 0x7f3cfc144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 0x7f3cfc144fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41968/0 (socket says 172.21.15.50:41968) 2023-10-19T10:38:31.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 -- 172.21.15.50:0/2043158855 learned_addr learned my addr 172.21.15.50:0/2043158855 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:31.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d00e4c640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d01e4e640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3cfc15c950 0x7f3cfc0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 -- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3cfc15c950 msgr2=0x7f3cfc0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3cfc15c950 0x7f3cfc0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 -- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3cfc15c570 msgr2=0x7f3cfc0b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d0164d640 1 -- 172.21.15.50:0/2043158855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40093a0 con 0x7f3cfc06b760 2023-10-19T10:38:31.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.730+0000 7f3d01e4e640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3cfc15c950 0x7f3cfc0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:31.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.731+0000 7f3d0164d640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 0x7f3cfc144fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4000c00 tx=0x7f3cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.731+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4004030 con 0x7f3cfc06b760 2023-10-19T10:38:31.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.731+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf40041b0 con 0x7f3cfc06b760 2023-10-19T10:38:31.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.731+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc157170 con 0x7f3cfc06b760 2023-10-19T10:38:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.731+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4005990 con 0x7f3cfc06b760 2023-10-19T10:38:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.731+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc145520 con 0x7f3cfc06b760 2023-10-19T10:38:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.732+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc0c9990 con 0x7f3cfc06b760 2023-10-19T10:38:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.733+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3cf4032640 con 0x7f3cfc06b760 2023-10-19T10:38:31.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.733+0000 7f3cf9ffb640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3ce806a050 0x7f3ce806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.733+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3cf40aa1e0 con 0x7f3cfc06b760 2023-10-19T10:38:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.733+0000 7f3d00e4c640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3ce806a050 0x7f3ce806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:31.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.734+0000 7f3d00e4c640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3ce806a050 0x7f3ce806c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc072a30 tx=0x7f3cf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:31.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.737+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3cfc0c9990 con 0x7f3cfc06b760 2023-10-19T10:38:31.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.911+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3cfc06af60 con 0x7f3ce806a050 2023-10-19T10:38:31.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.918+0000 7f3cf9ffb640 1 -- 172.21.15.50:0/2043158855 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+298251 (secure 0 0 0) 0x7f3cf000b200 con 0x7f3ce806a050 2023-10-19T10:38:31.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:31.926 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:38:31.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.921+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3ce806a050 msgr2=0x7f3ce806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.921+0000 7f3d038b1640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3ce806a050 0x7f3ce806c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc072a30 tx=0x7f3cf0008040 comp rx=0 tx=0).stop 2023-10-19T10:38:31.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.921+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 msgr2=0x7f3cfc144fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:31.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.921+0000 7f3d038b1640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 0x7f3cfc144fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4000c00 tx=0x7f3cf40046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 shutdown_connections 2023-10-19T10:38:31.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3ce806a050 0x7f3ce806c510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3cfc15c950 0x7f3cfc0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3cfc15c570 0x7f3cfc0b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 --2- 172.21.15.50:0/2043158855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cfc06b760 0x7f3cfc144fe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 >> 172.21.15.50:0/2043158855 conn(0x7f3cfc089ff0 msgr2=0x7f3cfc15aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 shutdown_connections 2023-10-19T10:38:31.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:31.922+0000 7f3d038b1640 1 -- 172.21.15.50:0/2043158855 wait complete. 2023-10-19T10:38:31.935 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-10-19T10:38:30.489504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237156,"kb_used_data":4608,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737564,"statfs":{"total":773094113280,"available":772851265536,"internally_reserved":0,"allocated":4718592,"data_stored":2719326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576210+0000","last_change":"2023-10-19T10:38:27.576210+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576210+0000","last_undegraded":"2023-10-19T10:38:27.576210+0000","last_fullsized":"2023-10-19T10:38:27.576210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576153+0000","last_change":"2023-10-19T10:38:27.576153+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576153+0000","last_undegraded":"2023-10-19T10:38:27.576153+0000","last_fullsized":"2023-10-19T10:38:27.576153+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589712+0000","last_change":"2023-10-19T10:38:27.584383+0000","last_active":"2023-10-19T10:38:27.589712+0000","last_peered":"2023-10-19T10:38:27.589712+0000","last_clean":"2023-10-19T10:38:27.589712+0000","last_became_active":"2023-10-19T10:38:27.583945+0000","last_became_peered":"2023-10-19T10:38:27.583945+0000","last_unstale":"2023-10-19T10:38:27.589712+0000","last_undegraded":"2023-10-19T10:38:27.589712+0000","last_fullsized":"2023-10-19T10:38:27.589712+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:17:13.906630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575423+0000","last_change":"2023-10-19T10:38:27.575423+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575423+0000","last_undegraded":"2023-10-19T10:38:27.575423+0000","last_fullsized":"2023-10-19T10:38:27.575423+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.592015+0000","last_change":"2023-10-19T10:38:27.589861+0000","last_active":"2023-10-19T10:38:27.592015+0000","last_peered":"2023-10-19T10:38:27.592015+0000","last_clean":"2023-10-19T10:38:27.592015+0000","last_became_active":"2023-10-19T10:38:27.589594+0000","last_became_peered":"2023-10-19T10:38:27.589594+0000","last_unstale":"2023-10-19T10:38:27.592015+0000","last_undegraded":"2023-10-19T10:38:27.592015+0000","last_fullsized":"2023-10-19T10:38:27.592015+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:32:26.735762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575835+0000","last_change":"2023-10-19T10:38:27.575835+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575835+0000","last_undegraded":"2023-10-19T10:38:27.575835+0000","last_fullsized":"2023-10-19T10:38:27.575835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589683+0000","last_change":"2023-10-19T10:38:27.584475+0000","last_active":"2023-10-19T10:38:27.589683+0000","last_peered":"2023-10-19T10:38:27.589683+0000","last_clean":"2023-10-19T10:38:27.589683+0000","last_became_active":"2023-10-19T10:38:27.584175+0000","last_became_peered":"2023-10-19T10:38:27.584175+0000","last_unstale":"2023-10-19T10:38:27.589683+0000","last_undegraded":"2023-10-19T10:38:27.589683+0000","last_fullsized":"2023-10-19T10:38:27.589683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:45:33.090392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575636+0000","last_change":"2023-10-19T10:38:27.575636+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575636+0000","last_undegraded":"2023-10-19T10:38:27.575636+0000","last_fullsized":"2023-10-19T10:38:27.575636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.588906+0000","last_change":"2023-10-19T10:38:27.584351+0000","last_active":"2023-10-19T10:38:27.588906+0000","last_peered":"2023-10-19T10:38:27.588906+0000","last_clean":"2023-10-19T10:38:27.588906+0000","last_became_active":"2023-10-19T10:38:27.583963+0000","last_became_peered":"2023-10-19T10:38:27.583963+0000","last_unstale":"2023-10-19T10:38:27.588906+0000","last_undegraded":"2023-10-19T10:38:27.588906+0000","last_fullsized":"2023-10-19T10:38:27.588906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:16.906120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594279+0000","last_change":"2023-10-19T10:38:27.590766+0000","last_active":"2023-10-19T10:38:27.594279+0000","last_peered":"2023-10-19T10:38:27.594279+0000","last_clean":"2023-10-19T10:38:27.594279+0000","last_became_active":"2023-10-19T10:38:27.590488+0000","last_became_peered":"2023-10-19T10:38:27.590488+0000","last_unstale":"2023-10-19T10:38:27.594279+0000","last_undegraded":"2023-10-19T10:38:27.594279+0000","last_fullsized":"2023-10-19T10:38:27.594279+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:19:14.891257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.124735+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:38:22.124735+0000","last_peered":"2023-10-19T10:38:22.124735+0000","last_clean":"2023-10-19T10:38:22.124735+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:38:22.124735+0000","last_undegraded":"2023-10-19T10:38:22.124735+0000","last_fullsized":"2023-10-19T10:38:22.124735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:53:23.212478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655521+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:38:22.655521+0000","last_peered":"2023-10-19T10:38:22.655521+0000","last_clean":"2023-10-19T10:38:22.655521+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:38:22.655521+0000","last_undegraded":"2023-10-19T10:38:22.655521+0000","last_fullsized":"2023-10-19T10:38:22.655521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:33:52.868361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.578327+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:38:27.578327+0000","last_peered":"2023-10-19T10:38:27.578327+0000","last_clean":"2023-10-19T10:38:27.578327+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:38:27.578327+0000","last_undegraded":"2023-10-19T10:38:27.578327+0000","last_fullsized":"2023-10-19T10:38:27.578327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:15.047277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655464+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:38:22.655464+0000","last_peered":"2023-10-19T10:38:22.655464+0000","last_clean":"2023-10-19T10:38:22.655464+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:38:22.655464+0000","last_undegraded":"2023-10-19T10:38:22.655464+0000","last_fullsized":"2023-10-19T10:38:22.655464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:50:47.137818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655692+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:38:22.655692+0000","last_peered":"2023-10-19T10:38:22.655692+0000","last_clean":"2023-10-19T10:38:22.655692+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:38:22.655692+0000","last_undegraded":"2023-10-19T10:38:22.655692+0000","last_fullsized":"2023-10-19T10:38:22.655692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:06:59.602523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594316+0000","last_change":"2023-10-19T10:38:27.590779+0000","last_active":"2023-10-19T10:38:27.594316+0000","last_peered":"2023-10-19T10:38:27.594316+0000","last_clean":"2023-10-19T10:38:27.594316+0000","last_became_active":"2023-10-19T10:38:27.590482+0000","last_became_peered":"2023-10-19T10:38:27.590482+0000","last_unstale":"2023-10-19T10:38:27.594316+0000","last_undegraded":"2023-10-19T10:38:27.594316+0000","last_fullsized":"2023-10-19T10:38:27.594316+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:10:42.478900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.123623+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:38:22.123623+0000","last_peered":"2023-10-19T10:38:22.123623+0000","last_clean":"2023-10-19T10:38:22.123623+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:38:22.123623+0000","last_undegraded":"2023-10-19T10:38:22.123623+0000","last_fullsized":"2023-10-19T10:38:22.123623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:13:16.360504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575245+0000","last_change":"2023-10-19T10:38:27.575245+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575245+0000","last_undegraded":"2023-10-19T10:38:27.575245+0000","last_fullsized":"2023-10-19T10:38:27.575245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.576832+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:38:27.576832+0000","last_peered":"2023-10-19T10:38:27.576832+0000","last_clean":"2023-10-19T10:38:27.576832+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:38:27.576832+0000","last_undegraded":"2023-10-19T10:38:27.576832+0000","last_fullsized":"2023-10-19T10:38:27.576832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:43:37.587705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655688+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:38:22.655688+0000","last_peered":"2023-10-19T10:38:22.655688+0000","last_clean":"2023-10-19T10:38:22.655688+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:38:22.655688+0000","last_undegraded":"2023-10-19T10:38:22.655688+0000","last_fullsized":"2023-10-19T10:38:22.655688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:47:46.186082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.577404+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:38:27.577404+0000","last_peered":"2023-10-19T10:38:27.577404+0000","last_clean":"2023-10-19T10:38:27.577404+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:38:27.577404+0000","last_undegraded":"2023-10-19T10:38:27.577404+0000","last_fullsized":"2023-10-19T10:38:27.577404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:45.342865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":794443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.501}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.377},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.438}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.441},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.505}]},{"osd":2,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.754},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.752},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":781013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.307}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.374}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.621}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:38:31.941 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:38:32.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.079+0000 7f76a191f640 1 Processor -- start 2023-10-19T10:38:32.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.079+0000 7f76a191f640 1 -- start start 2023-10-19T10:38:32.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f76a191f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f769c15c570 0x7f769c0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f76a191f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f769c15c950 0x7f769c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f76a191f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0cde60 con 0x7f769c06b760 2023-10-19T10:38:32.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f76a191f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c06aa30 con 0x7f769c15c950 2023-10-19T10:38:32.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f76a191f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c06b2d0 con 0x7f769c15c570 2023-10-19T10:38:32.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f769c15c950 0x7f769c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f769c15c950 0x7f769c0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41982/0 (socket says 172.21.15.50:41982) 2023-10-19T10:38:32.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 -- 172.21.15.50:0/3325112898 learned_addr learned my addr 172.21.15.50:0/3325112898 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:32.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769affd640 1 --2- 172.21.15.50:0/3325112898 >> v2:172.21.15.183:3300/0 conn(0x7f769c15c570 0x7f769c0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 -- 172.21.15.50:0/3325112898 >> v2:172.21.15.183:3300/0 conn(0x7f769c15c570 msgr2=0x7f769c0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:32.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 --2- 172.21.15.50:0/3325112898 >> v2:172.21.15.183:3300/0 conn(0x7f769c15c570 0x7f769c0d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 -- 172.21.15.50:0/3325112898 >> v1:172.21.15.183:6789/0 conn(0x7f769c06b760 legacy=0x7f769c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:32.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769a7fc640 1 -- 172.21.15.50:0/3325112898 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0ce090 con 0x7f769c15c950 2023-10-19T10:38:32.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.080+0000 7f769affd640 1 --2- 172.21.15.50:0/3325112898 >> v2:172.21.15.183:3300/0 conn(0x7f769c15c570 0x7f769c0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:32.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.081+0000 7f769a7fc640 1 --2- 172.21.15.50:0/3325112898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c0cd920 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f769c0d2670 tx=0x7f768402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b974d3ba64a794 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.081+0000 7f7699ffb640 1 -- 172.21.15.50:0/3325112898 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840317e0 con 0x7f769c15c950 2023-10-19T10:38:32.087 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.081+0000 7f7699ffb640 1 -- 172.21.15.50:0/3325112898 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7684031940 con 0x7f769c15c950 2023-10-19T10:38:32.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.081+0000 7f7699ffb640 1 -- 172.21.15.50:0/3325112898 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7684031cd0 con 0x7f769c15c950 2023-10-19T10:38:32.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 -- 172.21.15.50:0/3325112898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 msgr2=0x7f769c0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 --2- 172.21.15.50:0/3325112898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c0cd920 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f769c0d2670 tx=0x7f768402b880 comp rx=0 tx=0).stop 2023-10-19T10:38:32.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 -- 172.21.15.50:0/3325112898 shutdown_connections 2023-10-19T10:38:32.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 --2- 172.21.15.50:0/3325112898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c0cd920 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 --2- 172.21.15.50:0/3325112898 >> v2:172.21.15.183:3300/0 conn(0x7f769c15c570 0x7f769c0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 -- 172.21.15.50:0/3325112898 >> 172.21.15.50:0/3325112898 conn(0x7f769c089ff0 msgr2=0x7f769c09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:32.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 -- 172.21.15.50:0/3325112898 shutdown_connections 2023-10-19T10:38:32.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.082+0000 7f76a191f640 1 -- 172.21.15.50:0/3325112898 wait complete. 2023-10-19T10:38:32.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.083+0000 7f76a191f640 1 Processor -- start 2023-10-19T10:38:32.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.083+0000 7f76a191f640 1 -- start start 2023-10-19T10:38:32.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f76a191f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769c06b760 0x7f769c1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f76a191f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769c15c570 0x7f769c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f76a191f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f76a191f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0c75c0 con 0x7f769c15c950 2023-10-19T10:38:32.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f76a191f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c1f2a40 con 0x7f769c15c570 2023-10-19T10:38:32.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f76a191f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0c6240 con 0x7f769c06b760 2023-10-19T10:38:32.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41988/0 (socket says 172.21.15.50:41988) 2023-10-19T10:38:32.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 -- 172.21.15.50:0/3303829518 learned_addr learned my addr 172.21.15.50:0/3303829518 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:32.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769a7fc640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769c15c570 0x7f769c159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769affd640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769c06b760 0x7f769c1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 -- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769c06b760 msgr2=0x7f769c1f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769c06b760 0x7f769c1f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 -- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769c15c570 msgr2=0x7f769c159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769c15c570 0x7f769c159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 -- 172.21.15.50:0/3303829518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840094d0 con 0x7f769c15c950 2023-10-19T10:38:32.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769a7fc640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769c15c570 0x7f769c159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:32.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769affd640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769c06b760 0x7f769c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:32.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.084+0000 7f769b7fe640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c160480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b9e0 tx=0x7f768c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.085+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c003b70 con 0x7f769c15c950 2023-10-19T10:38:32.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.085+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c003cd0 con 0x7f769c15c950 2023-10-19T10:38:32.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.085+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c0c6520 con 0x7f769c15c950 2023-10-19T10:38:32.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.085+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c00ca80 con 0x7f769c15c950 2023-10-19T10:38:32.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.085+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c0c6ae0 con 0x7f769c15c950 2023-10-19T10:38:32.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.086+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c0b09d0 con 0x7f769c15c950 2023-10-19T10:38:32.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.086+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f768c00cbe0 con 0x7f769c15c950 2023-10-19T10:38:32.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.086+0000 7f7693fff640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f767c06a050 0x7f767c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.087+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f768c0883a0 con 0x7f769c15c950 2023-10-19T10:38:32.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.087+0000 7f769affd640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f767c06a050 0x7f767c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.087+0000 7f769affd640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f767c06a050 0x7f767c06c510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7694006fd0 tx=0x7f7694008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.091+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f768c053960 con 0x7f769c15c950 2023-10-19T10:38:32.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.263+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f769c06af60 con 0x7f767c06a050 2023-10-19T10:38:32.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.271+0000 7f7693fff640 1 -- 172.21.15.50:0/3303829518 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+298251 (secure 0 0 0) 0x7f769400a7d0 con 0x7f767c06a050 2023-10-19T10:38:32.277 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:32.278 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:38:32.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.274+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f767c06a050 msgr2=0x7f767c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.274+0000 7f76a191f640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f767c06a050 0x7f767c06c510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7694006fd0 tx=0x7f7694008040 comp rx=0 tx=0).stop 2023-10-19T10:38:32.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.274+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 msgr2=0x7f769c160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.274+0000 7f76a191f640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c160480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b9e0 tx=0x7f768c00bea0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 shutdown_connections 2023-10-19T10:38:32.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f767c06a050 0x7f767c06c510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f769c15c950 0x7f769c160480 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769c15c570 0x7f769c159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 --2- 172.21.15.50:0/3303829518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769c06b760 0x7f769c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 >> 172.21.15.50:0/3303829518 conn(0x7f769c089ff0 msgr2=0x7f769c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:32.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 shutdown_connections 2023-10-19T10:38:32.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.275+0000 7f76a191f640 1 -- 172.21.15.50:0/3303829518 wait complete. 2023-10-19T10:38:32.288 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-10-19T10:38:30.489504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237156,"kb_used_data":4608,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737564,"statfs":{"total":773094113280,"available":772851265536,"internally_reserved":0,"allocated":4718592,"data_stored":2719326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576210+0000","last_change":"2023-10-19T10:38:27.576210+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576210+0000","last_undegraded":"2023-10-19T10:38:27.576210+0000","last_fullsized":"2023-10-19T10:38:27.576210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576153+0000","last_change":"2023-10-19T10:38:27.576153+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576153+0000","last_undegraded":"2023-10-19T10:38:27.576153+0000","last_fullsized":"2023-10-19T10:38:27.576153+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589712+0000","last_change":"2023-10-19T10:38:27.584383+0000","last_active":"2023-10-19T10:38:27.589712+0000","last_peered":"2023-10-19T10:38:27.589712+0000","last_clean":"2023-10-19T10:38:27.589712+0000","last_became_active":"2023-10-19T10:38:27.583945+0000","last_became_peered":"2023-10-19T10:38:27.583945+0000","last_unstale":"2023-10-19T10:38:27.589712+0000","last_undegraded":"2023-10-19T10:38:27.589712+0000","last_fullsized":"2023-10-19T10:38:27.589712+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:17:13.906630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575423+0000","last_change":"2023-10-19T10:38:27.575423+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575423+0000","last_undegraded":"2023-10-19T10:38:27.575423+0000","last_fullsized":"2023-10-19T10:38:27.575423+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.592015+0000","last_change":"2023-10-19T10:38:27.589861+0000","last_active":"2023-10-19T10:38:27.592015+0000","last_peered":"2023-10-19T10:38:27.592015+0000","last_clean":"2023-10-19T10:38:27.592015+0000","last_became_active":"2023-10-19T10:38:27.589594+0000","last_became_peered":"2023-10-19T10:38:27.589594+0000","last_unstale":"2023-10-19T10:38:27.592015+0000","last_undegraded":"2023-10-19T10:38:27.592015+0000","last_fullsized":"2023-10-19T10:38:27.592015+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:32:26.735762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575835+0000","last_change":"2023-10-19T10:38:27.575835+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575835+0000","last_undegraded":"2023-10-19T10:38:27.575835+0000","last_fullsized":"2023-10-19T10:38:27.575835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589683+0000","last_change":"2023-10-19T10:38:27.584475+0000","last_active":"2023-10-19T10:38:27.589683+0000","last_peered":"2023-10-19T10:38:27.589683+0000","last_clean":"2023-10-19T10:38:27.589683+0000","last_became_active":"2023-10-19T10:38:27.584175+0000","last_became_peered":"2023-10-19T10:38:27.584175+0000","last_unstale":"2023-10-19T10:38:27.589683+0000","last_undegraded":"2023-10-19T10:38:27.589683+0000","last_fullsized":"2023-10-19T10:38:27.589683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:45:33.090392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575636+0000","last_change":"2023-10-19T10:38:27.575636+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575636+0000","last_undegraded":"2023-10-19T10:38:27.575636+0000","last_fullsized":"2023-10-19T10:38:27.575636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.588906+0000","last_change":"2023-10-19T10:38:27.584351+0000","last_active":"2023-10-19T10:38:27.588906+0000","last_peered":"2023-10-19T10:38:27.588906+0000","last_clean":"2023-10-19T10:38:27.588906+0000","last_became_active":"2023-10-19T10:38:27.583963+0000","last_became_peered":"2023-10-19T10:38:27.583963+0000","last_unstale":"2023-10-19T10:38:27.588906+0000","last_undegraded":"2023-10-19T10:38:27.588906+0000","last_fullsized":"2023-10-19T10:38:27.588906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:16.906120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594279+0000","last_change":"2023-10-19T10:38:27.590766+0000","last_active":"2023-10-19T10:38:27.594279+0000","last_peered":"2023-10-19T10:38:27.594279+0000","last_clean":"2023-10-19T10:38:27.594279+0000","last_became_active":"2023-10-19T10:38:27.590488+0000","last_became_peered":"2023-10-19T10:38:27.590488+0000","last_unstale":"2023-10-19T10:38:27.594279+0000","last_undegraded":"2023-10-19T10:38:27.594279+0000","last_fullsized":"2023-10-19T10:38:27.594279+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:19:14.891257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.124735+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:38:22.124735+0000","last_peered":"2023-10-19T10:38:22.124735+0000","last_clean":"2023-10-19T10:38:22.124735+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:38:22.124735+0000","last_undegraded":"2023-10-19T10:38:22.124735+0000","last_fullsized":"2023-10-19T10:38:22.124735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:53:23.212478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655521+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:38:22.655521+0000","last_peered":"2023-10-19T10:38:22.655521+0000","last_clean":"2023-10-19T10:38:22.655521+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:38:22.655521+0000","last_undegraded":"2023-10-19T10:38:22.655521+0000","last_fullsized":"2023-10-19T10:38:22.655521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:33:52.868361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.578327+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:38:27.578327+0000","last_peered":"2023-10-19T10:38:27.578327+0000","last_clean":"2023-10-19T10:38:27.578327+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:38:27.578327+0000","last_undegraded":"2023-10-19T10:38:27.578327+0000","last_fullsized":"2023-10-19T10:38:27.578327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:15.047277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655464+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:38:22.655464+0000","last_peered":"2023-10-19T10:38:22.655464+0000","last_clean":"2023-10-19T10:38:22.655464+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:38:22.655464+0000","last_undegraded":"2023-10-19T10:38:22.655464+0000","last_fullsized":"2023-10-19T10:38:22.655464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:50:47.137818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655692+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:38:22.655692+0000","last_peered":"2023-10-19T10:38:22.655692+0000","last_clean":"2023-10-19T10:38:22.655692+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:38:22.655692+0000","last_undegraded":"2023-10-19T10:38:22.655692+0000","last_fullsized":"2023-10-19T10:38:22.655692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:06:59.602523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594316+0000","last_change":"2023-10-19T10:38:27.590779+0000","last_active":"2023-10-19T10:38:27.594316+0000","last_peered":"2023-10-19T10:38:27.594316+0000","last_clean":"2023-10-19T10:38:27.594316+0000","last_became_active":"2023-10-19T10:38:27.590482+0000","last_became_peered":"2023-10-19T10:38:27.590482+0000","last_unstale":"2023-10-19T10:38:27.594316+0000","last_undegraded":"2023-10-19T10:38:27.594316+0000","last_fullsized":"2023-10-19T10:38:27.594316+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:10:42.478900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.123623+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:38:22.123623+0000","last_peered":"2023-10-19T10:38:22.123623+0000","last_clean":"2023-10-19T10:38:22.123623+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:38:22.123623+0000","last_undegraded":"2023-10-19T10:38:22.123623+0000","last_fullsized":"2023-10-19T10:38:22.123623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:13:16.360504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575245+0000","last_change":"2023-10-19T10:38:27.575245+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575245+0000","last_undegraded":"2023-10-19T10:38:27.575245+0000","last_fullsized":"2023-10-19T10:38:27.575245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.576832+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:38:27.576832+0000","last_peered":"2023-10-19T10:38:27.576832+0000","last_clean":"2023-10-19T10:38:27.576832+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:38:27.576832+0000","last_undegraded":"2023-10-19T10:38:27.576832+0000","last_fullsized":"2023-10-19T10:38:27.576832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:43:37.587705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:27.561034+0000","last_change":"2023-10-19T10:38:27.561034+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.561034+0000","last_undegraded":"2023-10-19T10:38:27.561034+0000","last_fullsized":"2023-10-19T10:38:27.561034+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-19T10:38:26.552965+0000","last_change":"2023-10-19T10:38:26.552965+0000","last_active":"2023-10-19T10:38:26.552965+0000","last_peered":"2023-10-19T10:38:26.552965+0000","last_clean":"2023-10-19T10:38:26.552965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:26.552965+0000","last_undegraded":"2023-10-19T10:38:26.552965+0000","last_fullsized":"2023-10-19T10:38:26.552965+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-19T10:38:22.655688+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:38:22.655688+0000","last_peered":"2023-10-19T10:38:22.655688+0000","last_clean":"2023-10-19T10:38:22.655688+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:38:22.655688+0000","last_undegraded":"2023-10-19T10:38:22.655688+0000","last_fullsized":"2023-10-19T10:38:22.655688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:47:46.186082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.577404+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:38:27.577404+0000","last_peered":"2023-10-19T10:38:27.577404+0000","last_clean":"2023-10-19T10:38:27.577404+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:38:27.577404+0000","last_undegraded":"2023-10-19T10:38:27.577404+0000","last_fullsized":"2023-10-19T10:38:27.577404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:45.342865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":794443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.501}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.377},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.438}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.441},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.505}]},{"osd":2,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.754},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.752},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":438272,"data_stored":190645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1032192,"data_stored":781013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.307}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.374}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.621}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:38:32.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-10-19T10:38:32.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2023-10-19T10:38:27.576210+0000', 'last_change': '2023-10-19T10:38:27.576210+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.576210+0000', 'last_undegraded': '2023-10-19T10:38:27.576210+0000', 'last_fullsized': '2023-10-19T10:38:27.576210+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-19T10:38:32.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-10-19T10:38:32.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-10-19T10:38:32.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-10-19T10:38:32.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-10-19T10:38:32.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-10-19T10:38:32.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-10-19T10:38:32.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2023-10-19T10:38:27.576153+0000', 'last_change': '2023-10-19T10:38:27.576153+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.576153+0000', 'last_undegraded': '2023-10-19T10:38:27.576153+0000', 'last_fullsized': '2023-10-19T10:38:27.576153+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-19T10:38:32.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-10-19T10:38:32.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-10-19T10:38:32.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-10-19T10:38:32.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-10-19T10:38:32.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-10-19T10:38:32.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-10-19T10:38:32.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-10-19T10:38:32.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-10-19T10:38:32.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-10-19T10:38:32.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-10-19T10:38:32.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-10-19T10:38:32.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-10-19T10:38:32.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-10-19T10:38:32.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-10-19T10:38:32.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-10-19T10:38:32.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-10-19T10:38:32.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-10-19T10:38:32.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-10-19T10:38:32.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-10-19T10:38:32.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-10-19T10:38:32.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-10-19T10:38:32.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-10-19T10:38:32.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-10-19T10:38:32.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-10-19T10:38:32.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-10-19T10:38:32.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-10-19T10:38:32.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-10-19T10:38:32.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-10-19T10:38:32.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-10-19T10:38:32.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-10-19T10:38:32.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-10-19T10:38:32.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-10-19T10:38:32.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-10-19T10:38:32.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-10-19T10:38:32.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-10-19T10:38:32.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-10-19T10:38:32.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-10-19T10:38:32.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-10-19T10:38:32.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-10-19T10:38:32.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-10-19T10:38:32.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-10-19T10:38:32.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-10-19T10:38:32.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-10-19T10:38:32.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2023-10-19T10:38:27.575423+0000', 'last_change': '2023-10-19T10:38:27.575423+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.575423+0000', 'last_undegraded': '2023-10-19T10:38:27.575423+0000', 'last_fullsized': '2023-10-19T10:38:27.575423+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-19T10:38:32.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-10-19T10:38:32.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-10-19T10:38:32.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-10-19T10:38:32.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2023-10-19T10:38:27.575835+0000', 'last_change': '2023-10-19T10:38:27.575835+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.575835+0000', 'last_undegraded': '2023-10-19T10:38:27.575835+0000', 'last_fullsized': '2023-10-19T10:38:27.575835+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-19T10:38:32.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-10-19T10:38:32.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-10-19T10:38:32.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-10-19T10:38:32.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-10-19T10:38:32.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-10-19T10:38:32.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-10-19T10:38:32.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-10-19T10:38:32.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2023-10-19T10:38:27.575636+0000', 'last_change': '2023-10-19T10:38:27.575636+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.575636+0000', 'last_undegraded': '2023-10-19T10:38:27.575636+0000', 'last_fullsized': '2023-10-19T10:38:27.575636+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-19T10:38:32.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-10-19T10:38:32.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-10-19T10:38:32.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-10-19T10:38:32.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-10-19T10:38:32.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-10-19T10:38:32.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-10-19T10:38:32.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-10-19T10:38:32.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-10-19T10:38:32.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-10-19T10:38:32.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-10-19T10:38:32.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-10-19T10:38:32.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-10-19T10:38:32.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-10-19T10:38:32.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-10-19T10:38:32.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-10-19T10:38:32.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-10-19T10:38:32.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-10-19T10:38:32.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-10-19T10:38:32.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-10-19T10:38:32.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-10-19T10:38:32.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-10-19T10:38:32.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-10-19T10:38:32.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-10-19T10:38:32.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-10-19T10:38:32.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-10-19T10:38:32.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-10-19T10:38:32.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-10-19T10:38:32.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2023-10-19T10:38:27.575245+0000', 'last_change': '2023-10-19T10:38:27.575245+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.575245+0000', 'last_undegraded': '2023-10-19T10:38:27.575245+0000', 'last_fullsized': '2023-10-19T10:38:27.575245+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-19T10:38:32.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-10-19T10:38:32.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-10-19T10:38:32.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:27.561034+0000', 'last_change': '2023-10-19T10:38:27.561034+0000', 'last_active': '2023-10-19T10:38:27.561034+0000', 'last_peered': '2023-10-19T10:38:27.561034+0000', 'last_clean': '2023-10-19T10:38:27.561034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:27.561034+0000', 'last_undegraded': '2023-10-19T10:38:27.561034+0000', 'last_fullsized': '2023-10-19T10:38:27.561034+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:27.561034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-10-19T10:38:32.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-19T10:38:26.552965+0000', 'last_change': '2023-10-19T10:38:26.552965+0000', 'last_active': '2023-10-19T10:38:26.552965+0000', 'last_peered': '2023-10-19T10:38:26.552965+0000', 'last_clean': '2023-10-19T10:38:26.552965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-19T10:38:26.552965+0000', 'last_undegraded': '2023-10-19T10:38:26.552965+0000', 'last_fullsized': '2023-10-19T10:38:26.552965+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'last_clean_scrub_stamp': '2023-10-19T10:38:26.552965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-19T10:38:32.365 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-19T10:38:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.467+0000 7f5372478640 1 Processor -- start 2023-10-19T10:38:32.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.467+0000 7f5372478640 1 -- start start 2023-10-19T10:38:32.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.468+0000 7f5372478640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.468+0000 7f5372478640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f536c0d1e80 0x7f536c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.468+0000 7f5372478640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0b66e0 con 0x7f536c0d08e0 2023-10-19T10:38:32.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.468+0000 7f5372478640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0cf750 con 0x7f536c0d1e80 2023-10-19T10:38:32.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.468+0000 7f5372478640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c15cd50 con 0x7f536c15c970 2023-10-19T10:38:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f5370a15640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f536c0d1e80 0x7f536c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c0cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:50406/0 (socket says 172.21.15.50:50406) 2023-10-19T10:38:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536b7fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f536c0d08e0 0x7f536c0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:55616/0 (socket says 172.21.15.50:55616) 2023-10-19T10:38:32.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 -- 172.21.15.50:0/1454393448 learned_addr learned my addr 172.21.15.50:0/1454393448 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:32.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536affd640 1 -- 172.21.15.50:0/1454393448 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532343377 0 0) 0x7f5360001110 con 0x7f536c0d08e0 2023-10-19T10:38:32.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 -- 172.21.15.50:0/1454393448 >> v2:172.21.15.183:3300/0 conn(0x7f536c0d1e80 msgr2=0x7f536c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:32.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 --2- 172.21.15.50:0/1454393448 >> v2:172.21.15.183:3300/0 conn(0x7f536c0d1e80 0x7f536c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 -- 172.21.15.50:0/1454393448 >> v1:172.21.15.183:6789/0 conn(0x7f536c0d08e0 legacy=0x7f536c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.469+0000 7f536bfff640 1 -- 172.21.15.50:0/1454393448 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0d0de0 con 0x7f536c15c970 2023-10-19T10:38:32.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.470+0000 7f536bfff640 1 --2- 172.21.15.50:0/1454393448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c0cecd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f535c009870 tx=0x7f535c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0fde74876e4456 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.470+0000 7f536affd640 1 -- 172.21.15.50:0/1454393448 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c030840 con 0x7f536c15c970 2023-10-19T10:38:32.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.470+0000 7f536affd640 1 -- 172.21.15.50:0/1454393448 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c0309a0 con 0x7f536c15c970 2023-10-19T10:38:32.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.470+0000 7f536affd640 1 -- 172.21.15.50:0/1454393448 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c030d30 con 0x7f536c15c970 2023-10-19T10:38:32.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 -- 172.21.15.50:0/1454393448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 msgr2=0x7f536c0cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 --2- 172.21.15.50:0/1454393448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c0cecd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f535c009870 tx=0x7f535c02b980 comp rx=0 tx=0).stop 2023-10-19T10:38:32.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 -- 172.21.15.50:0/1454393448 shutdown_connections 2023-10-19T10:38:32.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 --2- 172.21.15.50:0/1454393448 >> v2:172.21.15.183:3300/0 conn(0x7f536c0d1e80 0x7f536c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 --2- 172.21.15.50:0/1454393448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c0cecd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 -- 172.21.15.50:0/1454393448 >> 172.21.15.50:0/1454393448 conn(0x7f536c06a4a0 msgr2=0x7f536c06a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:32.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 -- 172.21.15.50:0/1454393448 shutdown_connections 2023-10-19T10:38:32.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.471+0000 7f5372478640 1 -- 172.21.15.50:0/1454393448 wait complete. 2023-10-19T10:38:32.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.472+0000 7f5372478640 1 Processor -- start 2023-10-19T10:38:32.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.472+0000 7f5372478640 1 -- start start 2023-10-19T10:38:32.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5372478640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f536c0d08e0 0x7f536c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5372478640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 0x7f536c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5372478640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5372478640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c15fcc0 con 0x7f536c0d1e80 2023-10-19T10:38:32.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5372478640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c158a20 con 0x7f536c0d08e0 2023-10-19T10:38:32.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5372478640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c15dc60 con 0x7f536c15c970 2023-10-19T10:38:32.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 0x7f536c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 0x7f536c1584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41990/0 (socket says 172.21.15.50:41990) 2023-10-19T10:38:32.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 -- 172.21.15.50:0/635463596 learned_addr learned my addr 172.21.15.50:0/635463596 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:32.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536bfff640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f536c0d08e0 0x7f536c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f5370a15640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 -- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 msgr2=0x7f536c158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 -- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f536c0d08e0 msgr2=0x7f536c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f536c0d08e0 0x7f536c15ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 -- 172.21.15.50:0/635463596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0094d0 con 0x7f536c0d1e80 2023-10-19T10:38:32.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.473+0000 7f536b7fe640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 0x7f536c1584e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f536000c780 tx=0x7f536000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.474+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600072d0 con 0x7f536c0d1e80 2023-10-19T10:38:32.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.474+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5360007450 con 0x7f536c0d1e80 2023-10-19T10:38:32.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.474+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c15df40 con 0x7f536c0d1e80 2023-10-19T10:38:32.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.474+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600047e0 con 0x7f536c0d1e80 2023-10-19T10:38:32.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.474+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c15e500 con 0x7f536c0d1e80 2023-10-19T10:38:32.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.475+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c073990 con 0x7f536c0d1e80 2023-10-19T10:38:32.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.475+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5360004960 con 0x7f536c0d1e80 2023-10-19T10:38:32.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.475+0000 7f5368ff9640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f535806a050 0x7f535806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.475+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f53600877b0 con 0x7f536c0d1e80 2023-10-19T10:38:32.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.476+0000 7f536bfff640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f535806a050 0x7f535806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.476+0000 7f536bfff640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f535806a050 0x7f535806c510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f535c009870 tx=0x7f535c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.480+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5360052e10 con 0x7f536c0d1e80 2023-10-19T10:38:32.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.721+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f536c0b66e0 con 0x7f536c0d1e80 2023-10-19T10:38:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.722+0000 7f5368ff9640 1 -- 172.21.15.50:0/635463596 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1098 (secure 0 0 0) 0x7f536c0b66e0 con 0x7f536c0d1e80 2023-10-19T10:38:32.725 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:32.725 INFO:teuthology.orchestra.run.smithi050.stdout:{"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":122,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":42,"num_osds":8,"num_up_osds":8,"osd_up_since":1697711802,"num_in_osds":8,"osd_in_since":1697711792,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":99},{"state_name":"creating+peering","count":6}],"num_pgs":105,"num_pools":4,"num_objects":21,"data_bytes":592688,"bytes_used":243249152,"bytes_avail":772850864128,"bytes_total":773094113280,"inactive_pgs_ratio":0.05714285746216774,"write_bytes_sec":2246,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5775}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-19T10:38:02.482124+0000","services":{}},"progress_events":{}} 2023-10-19T10:38:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f535806a050 msgr2=0x7f535806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f535806a050 0x7f535806c510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f535c009870 tx=0x7f535c008030 comp rx=0 tx=0).stop 2023-10-19T10:38:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 msgr2=0x7f536c1584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 0x7f536c1584e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f536000c780 tx=0x7f536000cc40 comp rx=0 tx=0).stop 2023-10-19T10:38:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 shutdown_connections 2023-10-19T10:38:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f535806a050 0x7f535806c510 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f536c15c970 0x7f536c158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c0d1e80 0x7f536c1584e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 --2- 172.21.15.50:0/635463596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f536c0d08e0 0x7f536c15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 >> 172.21.15.50:0/635463596 conn(0x7f536c06a4a0 msgr2=0x7f536c0d2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:32.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 shutdown_connections 2023-10-19T10:38:32.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.724+0000 7f5372478640 1 -- 172.21.15.50:0/635463596 wait complete. 2023-10-19T10:38:32.737 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-19T10:38:32.737 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:38:32.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.841+0000 7fd6ad609640 1 Processor -- start 2023-10-19T10:38:32.844 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.841+0000 7fd6ad609640 1 -- start start 2023-10-19T10:38:32.845 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6ad609640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd6a815c570 0x7fd6a80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6ad609640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6ad609640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a80cde60 con 0x7fd6a806b760 2023-10-19T10:38:32.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6ad609640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a806aa30 con 0x7fd6a815c570 2023-10-19T10:38:32.846 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6ad609640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a806b2d0 con 0x7fd6a815c950 2023-10-19T10:38:32.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6a6ffd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd6a815c570 0x7fd6a80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6a67fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6a77fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fd6a806b760 0x7fd6a80d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:55620/0 (socket says 172.21.15.50:55620) 2023-10-19T10:38:32.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.842+0000 7fd6a77fe640 1 -- 172.21.15.50:0/924712554 learned_addr learned my addr 172.21.15.50:0/924712554 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:32.847 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.843+0000 7fd6a5ffb640 1 -- 172.21.15.50:0/924712554 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477285280 0 0) 0x7fd6a80cde60 con 0x7fd6a806b760 2023-10-19T10:38:32.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.843+0000 7fd6a5ffb640 1 -- 172.21.15.50:0/924712554 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd690003520 con 0x7fd6a806b760 2023-10-19T10:38:32.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.843+0000 7fd6a67fc640 1 -- 172.21.15.50:0/924712554 >> v2:172.21.15.183:3300/0 conn(0x7fd6a815c570 msgr2=0x7fd6a80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:32.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.843+0000 7fd6a67fc640 1 --2- 172.21.15.50:0/924712554 >> v2:172.21.15.183:3300/0 conn(0x7fd6a815c570 0x7fd6a80d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.843+0000 7fd6a67fc640 1 -- 172.21.15.50:0/924712554 >> v1:172.21.15.183:6789/0 conn(0x7fd6a806b760 legacy=0x7fd6a80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.848 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.843+0000 7fd6a67fc640 1 -- 172.21.15.50:0/924712554 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a80ce090 con 0x7fd6a815c950 2023-10-19T10:38:32.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6a67fc640 1 --2- 172.21.15.50:0/924712554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a80cd920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0097a0 tx=0x7fd69c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09f4a465110f25d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6a5ffb640 1 -- 172.21.15.50:0/924712554 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c02e070 con 0x7fd6a815c950 2023-10-19T10:38:32.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6a5ffb640 1 -- 172.21.15.50:0/924712554 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c0315a0 con 0x7fd6a815c950 2023-10-19T10:38:32.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6a5ffb640 1 -- 172.21.15.50:0/924712554 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c031910 con 0x7fd6a815c950 2023-10-19T10:38:32.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 -- 172.21.15.50:0/924712554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 msgr2=0x7fd6a80cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 --2- 172.21.15.50:0/924712554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a80cd920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0097a0 tx=0x7fd69c02b840 comp rx=0 tx=0).stop 2023-10-19T10:38:32.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 -- 172.21.15.50:0/924712554 shutdown_connections 2023-10-19T10:38:32.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 --2- 172.21.15.50:0/924712554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a80cd920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 --2- 172.21.15.50:0/924712554 >> v2:172.21.15.183:3300/0 conn(0x7fd6a815c570 0x7fd6a80d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 -- 172.21.15.50:0/924712554 >> 172.21.15.50:0/924712554 conn(0x7fd6a8089ff0 msgr2=0x7fd6a809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:32.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 -- 172.21.15.50:0/924712554 shutdown_connections 2023-10-19T10:38:32.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.844+0000 7fd6ad609640 1 -- 172.21.15.50:0/924712554 wait complete. 2023-10-19T10:38:32.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.845+0000 7fd6ad609640 1 Processor -- start 2023-10-19T10:38:32.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 -- start start 2023-10-19T10:38:32.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 0x7fd6a8150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a815c570 0x7fd6a8159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a80cc1b0 con 0x7fd6a806b760 2023-10-19T10:38:32.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a80cc490 con 0x7fd6a815c570 2023-10-19T10:38:32.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6ad609640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a81587a0 con 0x7fd6a815c950 2023-10-19T10:38:32.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6a6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 0x7fd6a8150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6a6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 0x7fd6a8150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41992/0 (socket says 172.21.15.50:41992) 2023-10-19T10:38:32.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.846+0000 7fd6a6ffd640 1 -- 172.21.15.50:0/1603974533 learned_addr learned my addr 172.21.15.50:0/1603974533 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:32.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a77fe640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a67fc640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a815c570 0x7fd6a8159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a6ffd640 1 -- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 msgr2=0x7fd6a815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a6ffd640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a6ffd640 1 -- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a815c570 msgr2=0x7fd6a8159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:32.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a6ffd640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a815c570 0x7fd6a8159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:32.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a6ffd640 1 -- 172.21.15.50:0/1603974533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c009400 con 0x7fd6a806b760 2023-10-19T10:38:32.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a67fc640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a815c570 0x7fd6a8159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:32.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6a6ffd640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 0x7fd6a8150950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000e900 tx=0x7fd6a000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0009660 con 0x7fd6a806b760 2023-10-19T10:38:32.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a000ca00 con 0x7fd6a806b760 2023-10-19T10:38:32.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a8158a80 con 0x7fd6a806b760 2023-10-19T10:38:32.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000cd70 con 0x7fd6a806b760 2023-10-19T10:38:32.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.847+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a8150e90 con 0x7fd6a806b760 2023-10-19T10:38:32.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.848+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a80cc600 con 0x7fd6a806b760 2023-10-19T10:38:32.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.848+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd6a0013470 con 0x7fd6a806b760 2023-10-19T10:38:32.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.849+0000 7fd68f7fe640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd69406a080 0x7fd69406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:32.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.849+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd6a0088cb0 con 0x7fd6a806b760 2023-10-19T10:38:32.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.849+0000 7fd6a67fc640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd69406a080 0x7fd69406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:32.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.850+0000 7fd6a67fc640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd69406a080 0x7fd69406c540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd69c036000 tx=0x7fd69c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:32.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:32.853+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd6a0054260 con 0x7fd6a806b760 2023-10-19T10:38:33.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.024+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6a806af60 con 0x7fd69406a080 2023-10-19T10:38:33.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.032+0000 7fd68f7fe640 1 -- 172.21.15.50:0/1603974533 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311185 (secure 0 0 0) 0x7fd69c004460 con 0x7fd69406a080 2023-10-19T10:38:33.035 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:38:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd69406a080 msgr2=0x7fd69406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd69406a080 0x7fd69406c540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd69c036000 tx=0x7fd69c008030 comp rx=0 tx=0).stop 2023-10-19T10:38:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 msgr2=0x7fd6a8150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 0x7fd6a8150950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000e900 tx=0x7fd6a000edc0 comp rx=0 tx=0).stop 2023-10-19T10:38:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 shutdown_connections 2023-10-19T10:38:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd69406a080 0x7fd69406c540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6a815c950 0x7fd6a815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a815c570 0x7fd6a8159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 --2- 172.21.15.50:0/1603974533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a806b760 0x7fd6a8150950 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 >> 172.21.15.50:0/1603974533 conn(0x7fd6a8089ff0 msgr2=0x7fd6a81540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 shutdown_connections 2023-10-19T10:38:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:33.035+0000 7fd6ad609640 1 -- 172.21.15.50:0/1603974533 wait complete. 2023-10-19T10:38:33.049 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-10-19T10:38:32.490002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237548,"kb_used_data":4980,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737172,"statfs":{"total":773094113280,"available":772850864128,"internally_reserved":0,"allocated":5099520,"data_stored":2905067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.925635"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576210+0000","last_change":"2023-10-19T10:38:27.576210+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576210+0000","last_undegraded":"2023-10-19T10:38:27.576210+0000","last_fullsized":"2023-10-19T10:38:27.576210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148738+0000","last_change":"2023-10-19T10:38:28.590368+0000","last_active":"2023-10-19T10:38:31.148738+0000","last_peered":"2023-10-19T10:38:31.148738+0000","last_clean":"2023-10-19T10:38:31.148738+0000","last_became_active":"2023-10-19T10:38:28.590116+0000","last_became_peered":"2023-10-19T10:38:28.590116+0000","last_unstale":"2023-10-19T10:38:31.148738+0000","last_undegraded":"2023-10-19T10:38:31.148738+0000","last_fullsized":"2023-10-19T10:38:31.148738+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:15:53.169196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714817+0000","last_change":"2023-10-19T10:38:28.596684+0000","last_active":"2023-10-19T10:38:30.714817+0000","last_peered":"2023-10-19T10:38:30.714817+0000","last_clean":"2023-10-19T10:38:30.714817+0000","last_became_active":"2023-10-19T10:38:28.596346+0000","last_became_peered":"2023-10-19T10:38:28.596346+0000","last_unstale":"2023-10-19T10:38:30.714817+0000","last_undegraded":"2023-10-19T10:38:30.714817+0000","last_fullsized":"2023-10-19T10:38:30.714817+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:15:12.478642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937673+0000","last_change":"2023-10-19T10:38:28.586191+0000","last_active":"2023-10-19T10:38:29.937673+0000","last_peered":"2023-10-19T10:38:29.937673+0000","last_clean":"2023-10-19T10:38:29.937673+0000","last_became_active":"2023-10-19T10:38:28.585819+0000","last_became_peered":"2023-10-19T10:38:28.585819+0000","last_unstale":"2023-10-19T10:38:29.937673+0000","last_undegraded":"2023-10-19T10:38:29.937673+0000","last_fullsized":"2023-10-19T10:38:29.937673+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:40.058837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148191+0000","last_change":"2023-10-19T10:38:28.587084+0000","last_active":"2023-10-19T10:38:31.148191+0000","last_peered":"2023-10-19T10:38:31.148191+0000","last_clean":"2023-10-19T10:38:31.148191+0000","last_became_active":"2023-10-19T10:38:28.586717+0000","last_became_peered":"2023-10-19T10:38:28.586717+0000","last_unstale":"2023-10-19T10:38:31.148191+0000","last_undegraded":"2023-10-19T10:38:31.148191+0000","last_fullsized":"2023-10-19T10:38:31.148191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:35:04.830904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592127+0000","last_change":"2023-10-19T10:38:28.591287+0000","last_active":"2023-10-19T10:38:29.592127+0000","last_peered":"2023-10-19T10:38:29.592127+0000","last_clean":"2023-10-19T10:38:29.592127+0000","last_became_active":"2023-10-19T10:38:28.590992+0000","last_became_peered":"2023-10-19T10:38:28.590992+0000","last_unstale":"2023-10-19T10:38:29.592127+0000","last_undegraded":"2023-10-19T10:38:29.592127+0000","last_fullsized":"2023-10-19T10:38:29.592127+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:14:35.055092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.576153+0000","last_change":"2023-10-19T10:38:27.576153+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.576153+0000","last_undegraded":"2023-10-19T10:38:27.576153+0000","last_fullsized":"2023-10-19T10:38:27.576153+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937431+0000","last_change":"2023-10-19T10:38:28.592196+0000","last_active":"2023-10-19T10:38:29.937431+0000","last_peered":"2023-10-19T10:38:29.937431+0000","last_clean":"2023-10-19T10:38:29.937431+0000","last_became_active":"2023-10-19T10:38:28.591908+0000","last_became_peered":"2023-10-19T10:38:28.591908+0000","last_unstale":"2023-10-19T10:38:29.937431+0000","last_undegraded":"2023-10-19T10:38:29.937431+0000","last_fullsized":"2023-10-19T10:38:29.937431+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:40:46.629010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938007+0000","last_change":"2023-10-19T10:38:28.607434+0000","last_active":"2023-10-19T10:38:29.938007+0000","last_peered":"2023-10-19T10:38:29.938007+0000","last_clean":"2023-10-19T10:38:29.938007+0000","last_became_active":"2023-10-19T10:38:28.607177+0000","last_became_peered":"2023-10-19T10:38:28.607177+0000","last_unstale":"2023-10-19T10:38:29.938007+0000","last_undegraded":"2023-10-19T10:38:29.938007+0000","last_fullsized":"2023-10-19T10:38:29.938007+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:57.487371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938012+0000","last_change":"2023-10-19T10:38:28.586306+0000","last_active":"2023-10-19T10:38:29.938012+0000","last_peered":"2023-10-19T10:38:29.938012+0000","last_clean":"2023-10-19T10:38:29.938012+0000","last_became_active":"2023-10-19T10:38:28.585875+0000","last_became_peered":"2023-10-19T10:38:28.585875+0000","last_unstale":"2023-10-19T10:38:29.938012+0000","last_undegraded":"2023-10-19T10:38:29.938012+0000","last_fullsized":"2023-10-19T10:38:29.938012+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:08:14.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937798+0000","last_change":"2023-10-19T10:38:28.607171+0000","last_active":"2023-10-19T10:38:29.937798+0000","last_peered":"2023-10-19T10:38:29.937798+0000","last_clean":"2023-10-19T10:38:29.937798+0000","last_became_active":"2023-10-19T10:38:28.606827+0000","last_became_peered":"2023-10-19T10:38:28.606827+0000","last_unstale":"2023-10-19T10:38:29.937798+0000","last_undegraded":"2023-10-19T10:38:29.937798+0000","last_fullsized":"2023-10-19T10:38:29.937798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:24.307807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148918+0000","last_change":"2023-10-19T10:38:28.591464+0000","last_active":"2023-10-19T10:38:31.148918+0000","last_peered":"2023-10-19T10:38:31.148918+0000","last_clean":"2023-10-19T10:38:31.148918+0000","last_became_active":"2023-10-19T10:38:28.591209+0000","last_became_peered":"2023-10-19T10:38:28.591209+0000","last_unstale":"2023-10-19T10:38:31.148918+0000","last_undegraded":"2023-10-19T10:38:31.148918+0000","last_fullsized":"2023-10-19T10:38:31.148918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:48:09.095111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589513+0000","last_change":"2023-10-19T10:38:28.601166+0000","last_active":"2023-10-19T10:38:29.589513+0000","last_peered":"2023-10-19T10:38:29.589513+0000","last_clean":"2023-10-19T10:38:29.589513+0000","last_became_active":"2023-10-19T10:38:28.600897+0000","last_became_peered":"2023-10-19T10:38:28.600897+0000","last_unstale":"2023-10-19T10:38:29.589513+0000","last_undegraded":"2023-10-19T10:38:29.589513+0000","last_fullsized":"2023-10-19T10:38:29.589513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:46:28.716699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937573+0000","last_change":"2023-10-19T10:38:28.597332+0000","last_active":"2023-10-19T10:38:29.937573+0000","last_peered":"2023-10-19T10:38:29.937573+0000","last_clean":"2023-10-19T10:38:29.937573+0000","last_became_active":"2023-10-19T10:38:28.597050+0000","last_became_peered":"2023-10-19T10:38:28.597050+0000","last_unstale":"2023-10-19T10:38:29.937573+0000","last_undegraded":"2023-10-19T10:38:29.937573+0000","last_fullsized":"2023-10-19T10:38:29.937573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:35:51.161014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592024+0000","last_change":"2023-10-19T10:38:28.588702+0000","last_active":"2023-10-19T10:38:29.592024+0000","last_peered":"2023-10-19T10:38:29.592024+0000","last_clean":"2023-10-19T10:38:29.592024+0000","last_became_active":"2023-10-19T10:38:28.588042+0000","last_became_peered":"2023-10-19T10:38:28.588042+0000","last_unstale":"2023-10-19T10:38:29.592024+0000","last_undegraded":"2023-10-19T10:38:29.592024+0000","last_fullsized":"2023-10-19T10:38:29.592024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:40:46.629010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592702+0000","last_change":"2023-10-19T10:38:28.592184+0000","last_active":"2023-10-19T10:38:29.592702+0000","last_peered":"2023-10-19T10:38:29.592702+0000","last_clean":"2023-10-19T10:38:29.592702+0000","last_became_active":"2023-10-19T10:38:28.591679+0000","last_became_peered":"2023-10-19T10:38:28.591679+0000","last_unstale":"2023-10-19T10:38:29.592702+0000","last_undegraded":"2023-10-19T10:38:29.592702+0000","last_fullsized":"2023-10-19T10:38:29.592702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:20:35.229068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715047+0000","last_change":"2023-10-19T10:38:28.588006+0000","last_active":"2023-10-19T10:38:30.715047+0000","last_peered":"2023-10-19T10:38:30.715047+0000","last_clean":"2023-10-19T10:38:30.715047+0000","last_became_active":"2023-10-19T10:38:28.587712+0000","last_became_peered":"2023-10-19T10:38:28.587712+0000","last_unstale":"2023-10-19T10:38:30.715047+0000","last_undegraded":"2023-10-19T10:38:30.715047+0000","last_fullsized":"2023-10-19T10:38:30.715047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:26:35.409939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714954+0000","last_change":"2023-10-19T10:38:28.601329+0000","last_active":"2023-10-19T10:38:30.714954+0000","last_peered":"2023-10-19T10:38:30.714954+0000","last_clean":"2023-10-19T10:38:30.714954+0000","last_became_active":"2023-10-19T10:38:28.600859+0000","last_became_peered":"2023-10-19T10:38:28.600859+0000","last_unstale":"2023-10-19T10:38:30.714954+0000","last_undegraded":"2023-10-19T10:38:30.714954+0000","last_fullsized":"2023-10-19T10:38:30.714954+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:16:56.445788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593221+0000","last_change":"2023-10-19T10:38:28.581733+0000","last_active":"2023-10-19T10:38:29.593221+0000","last_peered":"2023-10-19T10:38:29.593221+0000","last_clean":"2023-10-19T10:38:29.593221+0000","last_became_active":"2023-10-19T10:38:28.581421+0000","last_became_peered":"2023-10-19T10:38:28.581421+0000","last_unstale":"2023-10-19T10:38:29.593221+0000","last_undegraded":"2023-10-19T10:38:29.593221+0000","last_fullsized":"2023-10-19T10:38:29.593221+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:58:04.017833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589407+0000","last_change":"2023-10-19T10:38:28.586586+0000","last_active":"2023-10-19T10:38:29.589407+0000","last_peered":"2023-10-19T10:38:29.589407+0000","last_clean":"2023-10-19T10:38:29.589407+0000","last_became_active":"2023-10-19T10:38:28.586288+0000","last_became_peered":"2023-10-19T10:38:28.586288+0000","last_unstale":"2023-10-19T10:38:29.589407+0000","last_undegraded":"2023-10-19T10:38:29.589407+0000","last_fullsized":"2023-10-19T10:38:29.589407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:52:51.318834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592275+0000","last_change":"2023-10-19T10:38:28.601706+0000","last_active":"2023-10-19T10:38:29.592275+0000","last_peered":"2023-10-19T10:38:29.592275+0000","last_clean":"2023-10-19T10:38:29.592275+0000","last_became_active":"2023-10-19T10:38:28.601355+0000","last_became_peered":"2023-10-19T10:38:28.601355+0000","last_unstale":"2023-10-19T10:38:29.592275+0000","last_undegraded":"2023-10-19T10:38:29.592275+0000","last_fullsized":"2023-10-19T10:38:29.592275+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:35:51.161014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937711+0000","last_change":"2023-10-19T10:38:28.586233+0000","last_active":"2023-10-19T10:38:29.937711+0000","last_peered":"2023-10-19T10:38:29.937711+0000","last_clean":"2023-10-19T10:38:29.937711+0000","last_became_active":"2023-10-19T10:38:28.585930+0000","last_became_peered":"2023-10-19T10:38:28.585930+0000","last_unstale":"2023-10-19T10:38:29.937711+0000","last_undegraded":"2023-10-19T10:38:29.937711+0000","last_fullsized":"2023-10-19T10:38:29.937711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:48:53.289286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592726+0000","last_change":"2023-10-19T10:38:28.584879+0000","last_active":"2023-10-19T10:38:29.592726+0000","last_peered":"2023-10-19T10:38:29.592726+0000","last_clean":"2023-10-19T10:38:29.592726+0000","last_became_active":"2023-10-19T10:38:28.584623+0000","last_became_peered":"2023-10-19T10:38:28.584623+0000","last_unstale":"2023-10-19T10:38:29.592726+0000","last_undegraded":"2023-10-19T10:38:29.592726+0000","last_fullsized":"2023-10-19T10:38:29.592726+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:00:04.990551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593377+0000","last_change":"2023-10-19T10:38:28.586155+0000","last_active":"2023-10-19T10:38:29.593377+0000","last_peered":"2023-10-19T10:38:29.593377+0000","last_clean":"2023-10-19T10:38:29.593377+0000","last_became_active":"2023-10-19T10:38:28.585600+0000","last_became_peered":"2023-10-19T10:38:28.585600+0000","last_unstale":"2023-10-19T10:38:29.593377+0000","last_undegraded":"2023-10-19T10:38:29.593377+0000","last_fullsized":"2023-10-19T10:38:29.593377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:07:38.741917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593179+0000","last_change":"2023-10-19T10:38:28.600175+0000","last_active":"2023-10-19T10:38:29.593179+0000","last_peered":"2023-10-19T10:38:29.593179+0000","last_clean":"2023-10-19T10:38:29.593179+0000","last_became_active":"2023-10-19T10:38:28.599704+0000","last_became_peered":"2023-10-19T10:38:28.599704+0000","last_unstale":"2023-10-19T10:38:29.593179+0000","last_undegraded":"2023-10-19T10:38:29.593179+0000","last_fullsized":"2023-10-19T10:38:29.593179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:22:55.924502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589544+0000","last_change":"2023-10-19T10:38:28.589160+0000","last_active":"2023-10-19T10:38:29.589544+0000","last_peered":"2023-10-19T10:38:29.589544+0000","last_clean":"2023-10-19T10:38:29.589544+0000","last_became_active":"2023-10-19T10:38:28.588852+0000","last_became_peered":"2023-10-19T10:38:28.588852+0000","last_unstale":"2023-10-19T10:38:29.589544+0000","last_undegraded":"2023-10-19T10:38:29.589544+0000","last_fullsized":"2023-10-19T10:38:29.589544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:53:27.890189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714933+0000","last_change":"2023-10-19T10:38:28.587517+0000","last_active":"2023-10-19T10:38:30.714933+0000","last_peered":"2023-10-19T10:38:30.714933+0000","last_clean":"2023-10-19T10:38:30.714933+0000","last_became_active":"2023-10-19T10:38:28.587261+0000","last_became_peered":"2023-10-19T10:38:28.587261+0000","last_unstale":"2023-10-19T10:38:30.714933+0000","last_undegraded":"2023-10-19T10:38:30.714933+0000","last_fullsized":"2023-10-19T10:38:30.714933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:51.984473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148803+0000","last_change":"2023-10-19T10:38:28.602166+0000","last_active":"2023-10-19T10:38:31.148803+0000","last_peered":"2023-10-19T10:38:31.148803+0000","last_clean":"2023-10-19T10:38:31.148803+0000","last_became_active":"2023-10-19T10:38:28.601867+0000","last_became_peered":"2023-10-19T10:38:28.601867+0000","last_unstale":"2023-10-19T10:38:31.148803+0000","last_undegraded":"2023-10-19T10:38:31.148803+0000","last_fullsized":"2023-10-19T10:38:31.148803+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:56:16.974974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593093+0000","last_change":"2023-10-19T10:38:28.593225+0000","last_active":"2023-10-19T10:38:29.593093+0000","last_peered":"2023-10-19T10:38:29.593093+0000","last_clean":"2023-10-19T10:38:29.593093+0000","last_became_active":"2023-10-19T10:38:28.592959+0000","last_became_peered":"2023-10-19T10:38:28.592959+0000","last_unstale":"2023-10-19T10:38:29.593093+0000","last_undegraded":"2023-10-19T10:38:29.593093+0000","last_fullsized":"2023-10-19T10:38:29.593093+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:41:29.958037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714820+0000","last_change":"2023-10-19T10:38:28.588475+0000","last_active":"2023-10-19T10:38:30.714820+0000","last_peered":"2023-10-19T10:38:30.714820+0000","last_clean":"2023-10-19T10:38:30.714820+0000","last_became_active":"2023-10-19T10:38:28.588181+0000","last_became_peered":"2023-10-19T10:38:28.588181+0000","last_unstale":"2023-10-19T10:38:30.714820+0000","last_undegraded":"2023-10-19T10:38:30.714820+0000","last_fullsized":"2023-10-19T10:38:30.714820+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:47:09.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592226+0000","last_change":"2023-10-19T10:38:28.588873+0000","last_active":"2023-10-19T10:38:29.592226+0000","last_peered":"2023-10-19T10:38:29.592226+0000","last_clean":"2023-10-19T10:38:29.592226+0000","last_became_active":"2023-10-19T10:38:28.588411+0000","last_became_peered":"2023-10-19T10:38:28.588411+0000","last_unstale":"2023-10-19T10:38:29.592226+0000","last_undegraded":"2023-10-19T10:38:29.592226+0000","last_fullsized":"2023-10-19T10:38:29.592226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:25:06.197277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589140+0000","last_change":"2023-10-19T10:38:28.597732+0000","last_active":"2023-10-19T10:38:29.589140+0000","last_peered":"2023-10-19T10:38:29.589140+0000","last_clean":"2023-10-19T10:38:29.589140+0000","last_became_active":"2023-10-19T10:38:28.597472+0000","last_became_peered":"2023-10-19T10:38:28.597472+0000","last_unstale":"2023-10-19T10:38:29.589140+0000","last_undegraded":"2023-10-19T10:38:29.589140+0000","last_fullsized":"2023-10-19T10:38:29.589140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:06:10.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605630+0000","last_change":"2023-10-19T10:38:28.589568+0000","last_active":"2023-10-19T10:38:30.605630+0000","last_peered":"2023-10-19T10:38:30.605630+0000","last_clean":"2023-10-19T10:38:30.605630+0000","last_became_active":"2023-10-19T10:38:28.589294+0000","last_became_peered":"2023-10-19T10:38:28.589294+0000","last_unstale":"2023-10-19T10:38:30.605630+0000","last_undegraded":"2023-10-19T10:38:30.605630+0000","last_fullsized":"2023-10-19T10:38:30.605630+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:19:15.899326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592527+0000","last_change":"2023-10-19T10:38:27.590076+0000","last_active":"2023-10-19T10:38:29.592527+0000","last_peered":"2023-10-19T10:38:29.592527+0000","last_clean":"2023-10-19T10:38:29.592527+0000","last_became_active":"2023-10-19T10:38:27.589685+0000","last_became_peered":"2023-10-19T10:38:27.589685+0000","last_unstale":"2023-10-19T10:38:29.592527+0000","last_undegraded":"2023-10-19T10:38:29.592527+0000","last_fullsized":"2023-10-19T10:38:29.592527+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:12:20.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593371+0000","last_change":"2023-10-19T10:38:28.586452+0000","last_active":"2023-10-19T10:38:29.593371+0000","last_peered":"2023-10-19T10:38:29.593371+0000","last_clean":"2023-10-19T10:38:29.593371+0000","last_became_active":"2023-10-19T10:38:28.585845+0000","last_became_peered":"2023-10-19T10:38:28.585845+0000","last_unstale":"2023-10-19T10:38:29.593371+0000","last_undegraded":"2023-10-19T10:38:29.593371+0000","last_fullsized":"2023-10-19T10:38:29.593371+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:06:13.597070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715110+0000","last_change":"2023-10-19T10:38:27.583864+0000","last_active":"2023-10-19T10:38:30.715110+0000","last_peered":"2023-10-19T10:38:30.715110+0000","last_clean":"2023-10-19T10:38:30.715110+0000","last_became_active":"2023-10-19T10:38:27.583595+0000","last_became_peered":"2023-10-19T10:38:27.583595+0000","last_unstale":"2023-10-19T10:38:30.715110+0000","last_undegraded":"2023-10-19T10:38:30.715110+0000","last_fullsized":"2023-10-19T10:38:30.715110+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:48:27.881884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148955+0000","last_change":"2023-10-19T10:38:28.596911+0000","last_active":"2023-10-19T10:38:31.148955+0000","last_peered":"2023-10-19T10:38:31.148955+0000","last_clean":"2023-10-19T10:38:31.148955+0000","last_became_active":"2023-10-19T10:38:28.596560+0000","last_became_peered":"2023-10-19T10:38:28.596560+0000","last_unstale":"2023-10-19T10:38:31.148955+0000","last_undegraded":"2023-10-19T10:38:31.148955+0000","last_fullsized":"2023-10-19T10:38:31.148955+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:44:42.869570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605742+0000","last_change":"2023-10-19T10:38:27.585185+0000","last_active":"2023-10-19T10:38:30.605742+0000","last_peered":"2023-10-19T10:38:30.605742+0000","last_clean":"2023-10-19T10:38:30.605742+0000","last_became_active":"2023-10-19T10:38:27.584912+0000","last_became_peered":"2023-10-19T10:38:27.584912+0000","last_unstale":"2023-10-19T10:38:30.605742+0000","last_undegraded":"2023-10-19T10:38:30.605742+0000","last_fullsized":"2023-10-19T10:38:30.605742+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:17:13.906630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605781+0000","last_change":"2023-10-19T10:38:28.582163+0000","last_active":"2023-10-19T10:38:30.605781+0000","last_peered":"2023-10-19T10:38:30.605781+0000","last_clean":"2023-10-19T10:38:30.605781+0000","last_became_active":"2023-10-19T10:38:28.581862+0000","last_became_peered":"2023-10-19T10:38:28.581862+0000","last_unstale":"2023-10-19T10:38:30.605781+0000","last_undegraded":"2023-10-19T10:38:30.605781+0000","last_fullsized":"2023-10-19T10:38:30.605781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:10:43.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589712+0000","last_change":"2023-10-19T10:38:27.584383+0000","last_active":"2023-10-19T10:38:27.589712+0000","last_peered":"2023-10-19T10:38:27.589712+0000","last_clean":"2023-10-19T10:38:27.589712+0000","last_became_active":"2023-10-19T10:38:27.583945+0000","last_became_peered":"2023-10-19T10:38:27.583945+0000","last_unstale":"2023-10-19T10:38:27.589712+0000","last_undegraded":"2023-10-19T10:38:27.589712+0000","last_fullsized":"2023-10-19T10:38:27.589712+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:17:13.906630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714814+0000","last_change":"2023-10-19T10:38:28.590904+0000","last_active":"2023-10-19T10:38:30.714814+0000","last_peered":"2023-10-19T10:38:30.714814+0000","last_clean":"2023-10-19T10:38:30.714814+0000","last_became_active":"2023-10-19T10:38:28.590604+0000","last_became_peered":"2023-10-19T10:38:28.590604+0000","last_unstale":"2023-10-19T10:38:30.714814+0000","last_undegraded":"2023-10-19T10:38:30.714814+0000","last_fullsized":"2023-10-19T10:38:30.714814+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:18:22.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592407+0000","last_change":"2023-10-19T10:38:27.589945+0000","last_active":"2023-10-19T10:38:29.592407+0000","last_peered":"2023-10-19T10:38:29.592407+0000","last_clean":"2023-10-19T10:38:29.592407+0000","last_became_active":"2023-10-19T10:38:27.589196+0000","last_became_peered":"2023-10-19T10:38:27.589196+0000","last_unstale":"2023-10-19T10:38:29.592407+0000","last_undegraded":"2023-10-19T10:38:29.592407+0000","last_fullsized":"2023-10-19T10:38:29.592407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:23.299738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937558+0000","last_change":"2023-10-19T10:38:28.609017+0000","last_active":"2023-10-19T10:38:29.937558+0000","last_peered":"2023-10-19T10:38:29.937558+0000","last_clean":"2023-10-19T10:38:29.937558+0000","last_became_active":"2023-10-19T10:38:28.608742+0000","last_became_peered":"2023-10-19T10:38:28.608742+0000","last_unstale":"2023-10-19T10:38:29.937558+0000","last_undegraded":"2023-10-19T10:38:29.937558+0000","last_fullsized":"2023-10-19T10:38:29.937558+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:25:06.197277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715001+0000","last_change":"2023-10-19T10:38:27.581569+0000","last_active":"2023-10-19T10:38:30.715001+0000","last_peered":"2023-10-19T10:38:30.715001+0000","last_clean":"2023-10-19T10:38:30.715001+0000","last_became_active":"2023-10-19T10:38:27.581289+0000","last_became_peered":"2023-10-19T10:38:27.581289+0000","last_unstale":"2023-10-19T10:38:30.715001+0000","last_undegraded":"2023-10-19T10:38:30.715001+0000","last_fullsized":"2023-10-19T10:38:30.715001+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:15:51.525358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937519+0000","last_change":"2023-10-19T10:38:28.587724+0000","last_active":"2023-10-19T10:38:29.937519+0000","last_peered":"2023-10-19T10:38:29.937519+0000","last_clean":"2023-10-19T10:38:29.937519+0000","last_became_active":"2023-10-19T10:38:28.587464+0000","last_became_peered":"2023-10-19T10:38:28.587464+0000","last_unstale":"2023-10-19T10:38:29.937519+0000","last_undegraded":"2023-10-19T10:38:29.937519+0000","last_fullsized":"2023-10-19T10:38:29.937519+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:59:51.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148198+0000","last_change":"2023-10-19T10:38:27.587587+0000","last_active":"2023-10-19T10:38:31.148198+0000","last_peered":"2023-10-19T10:38:31.148198+0000","last_clean":"2023-10-19T10:38:31.148198+0000","last_became_active":"2023-10-19T10:38:27.587288+0000","last_became_peered":"2023-10-19T10:38:27.587288+0000","last_unstale":"2023-10-19T10:38:31.148198+0000","last_undegraded":"2023-10-19T10:38:31.148198+0000","last_fullsized":"2023-10-19T10:38:31.148198+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:49:09.429497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592648+0000","last_change":"2023-10-19T10:38:28.585499+0000","last_active":"2023-10-19T10:38:29.592648+0000","last_peered":"2023-10-19T10:38:29.592648+0000","last_clean":"2023-10-19T10:38:29.592648+0000","last_became_active":"2023-10-19T10:38:28.585259+0000","last_became_peered":"2023-10-19T10:38:28.585259+0000","last_unstale":"2023-10-19T10:38:29.592648+0000","last_undegraded":"2023-10-19T10:38:29.592648+0000","last_fullsized":"2023-10-19T10:38:29.592648+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:52:51.318834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937896+0000","last_change":"2023-10-19T10:38:27.584386+0000","last_active":"2023-10-19T10:38:29.937896+0000","last_peered":"2023-10-19T10:38:29.937896+0000","last_clean":"2023-10-19T10:38:29.937896+0000","last_became_active":"2023-10-19T10:38:27.583893+0000","last_became_peered":"2023-10-19T10:38:27.583893+0000","last_unstale":"2023-10-19T10:38:29.937896+0000","last_undegraded":"2023-10-19T10:38:29.937896+0000","last_fullsized":"2023-10-19T10:38:29.937896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:24:18.658643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714999+0000","last_change":"2023-10-19T10:38:28.601147+0000","last_active":"2023-10-19T10:38:30.714999+0000","last_peered":"2023-10-19T10:38:30.714999+0000","last_clean":"2023-10-19T10:38:30.714999+0000","last_became_active":"2023-10-19T10:38:28.600303+0000","last_became_peered":"2023-10-19T10:38:28.600303+0000","last_unstale":"2023-10-19T10:38:30.714999+0000","last_undegraded":"2023-10-19T10:38:30.714999+0000","last_fullsized":"2023-10-19T10:38:30.714999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:00:10.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715125+0000","last_change":"2023-10-19T10:38:27.582927+0000","last_active":"2023-10-19T10:38:30.715125+0000","last_peered":"2023-10-19T10:38:30.715125+0000","last_clean":"2023-10-19T10:38:30.715125+0000","last_became_active":"2023-10-19T10:38:27.582650+0000","last_became_peered":"2023-10-19T10:38:27.582650+0000","last_unstale":"2023-10-19T10:38:30.715125+0000","last_undegraded":"2023-10-19T10:38:30.715125+0000","last_fullsized":"2023-10-19T10:38:30.715125+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:44:23.815810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575423+0000","last_change":"2023-10-19T10:38:27.575423+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575423+0000","last_undegraded":"2023-10-19T10:38:27.575423+0000","last_fullsized":"2023-10-19T10:38:27.575423+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938089+0000","last_change":"2023-10-19T10:38:27.586098+0000","last_active":"2023-10-19T10:38:29.938089+0000","last_peered":"2023-10-19T10:38:29.938089+0000","last_clean":"2023-10-19T10:38:29.938089+0000","last_became_active":"2023-10-19T10:38:27.585600+0000","last_became_peered":"2023-10-19T10:38:27.585600+0000","last_unstale":"2023-10-19T10:38:29.938089+0000","last_undegraded":"2023-10-19T10:38:29.938089+0000","last_fullsized":"2023-10-19T10:38:29.938089+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:35:03.822835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592357+0000","last_change":"2023-10-19T10:38:28.603572+0000","last_active":"2023-10-19T10:38:29.592357+0000","last_peered":"2023-10-19T10:38:29.592357+0000","last_clean":"2023-10-19T10:38:29.592357+0000","last_became_active":"2023-10-19T10:38:28.603273+0000","last_became_peered":"2023-10-19T10:38:28.603273+0000","last_unstale":"2023-10-19T10:38:29.592357+0000","last_undegraded":"2023-10-19T10:38:29.592357+0000","last_fullsized":"2023-10-19T10:38:29.592357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:48:53.289286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.592015+0000","last_change":"2023-10-19T10:38:27.589861+0000","last_active":"2023-10-19T10:38:27.592015+0000","last_peered":"2023-10-19T10:38:27.592015+0000","last_clean":"2023-10-19T10:38:27.592015+0000","last_became_active":"2023-10-19T10:38:27.589594+0000","last_became_peered":"2023-10-19T10:38:27.589594+0000","last_unstale":"2023-10-19T10:38:27.592015+0000","last_undegraded":"2023-10-19T10:38:27.592015+0000","last_fullsized":"2023-10-19T10:38:27.592015+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:32:26.735762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575835+0000","last_change":"2023-10-19T10:38:27.575835+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575835+0000","last_undegraded":"2023-10-19T10:38:27.575835+0000","last_fullsized":"2023-10-19T10:38:27.575835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589438+0000","last_change":"2023-10-19T10:38:27.587308+0000","last_active":"2023-10-19T10:38:29.589438+0000","last_peered":"2023-10-19T10:38:29.589438+0000","last_clean":"2023-10-19T10:38:29.589438+0000","last_became_active":"2023-10-19T10:38:27.586928+0000","last_became_peered":"2023-10-19T10:38:27.586928+0000","last_unstale":"2023-10-19T10:38:29.589438+0000","last_undegraded":"2023-10-19T10:38:29.589438+0000","last_fullsized":"2023-10-19T10:38:29.589438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:20:34.220998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593428+0000","last_change":"2023-10-19T10:38:28.588339+0000","last_active":"2023-10-19T10:38:29.593428+0000","last_peered":"2023-10-19T10:38:29.593428+0000","last_clean":"2023-10-19T10:38:29.593428+0000","last_became_active":"2023-10-19T10:38:28.588090+0000","last_became_peered":"2023-10-19T10:38:28.588090+0000","last_unstale":"2023-10-19T10:38:29.593428+0000","last_undegraded":"2023-10-19T10:38:29.593428+0000","last_fullsized":"2023-10-19T10:38:29.593428+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:05:54.866959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592484+0000","last_change":"2023-10-19T10:38:27.588963+0000","last_active":"2023-10-19T10:38:29.592484+0000","last_peered":"2023-10-19T10:38:29.592484+0000","last_clean":"2023-10-19T10:38:29.592484+0000","last_became_active":"2023-10-19T10:38:27.588617+0000","last_became_peered":"2023-10-19T10:38:27.588617+0000","last_unstale":"2023-10-19T10:38:29.592484+0000","last_undegraded":"2023-10-19T10:38:29.592484+0000","last_fullsized":"2023-10-19T10:38:29.592484+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:24:18.658643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937651+0000","last_change":"2023-10-19T10:38:28.590252+0000","last_active":"2023-10-19T10:38:29.937651+0000","last_peered":"2023-10-19T10:38:29.937651+0000","last_clean":"2023-10-19T10:38:29.937651+0000","last_became_active":"2023-10-19T10:38:28.589980+0000","last_became_peered":"2023-10-19T10:38:28.589980+0000","last_unstale":"2023-10-19T10:38:29.937651+0000","last_undegraded":"2023-10-19T10:38:29.937651+0000","last_fullsized":"2023-10-19T10:38:29.937651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:53:14.764775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.589683+0000","last_change":"2023-10-19T10:38:27.584475+0000","last_active":"2023-10-19T10:38:27.589683+0000","last_peered":"2023-10-19T10:38:27.589683+0000","last_clean":"2023-10-19T10:38:27.589683+0000","last_became_active":"2023-10-19T10:38:27.584175+0000","last_became_peered":"2023-10-19T10:38:27.584175+0000","last_unstale":"2023-10-19T10:38:27.589683+0000","last_undegraded":"2023-10-19T10:38:27.589683+0000","last_fullsized":"2023-10-19T10:38:27.589683+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:45:33.090392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592024+0000","last_change":"2023-10-19T10:38:28.594638+0000","last_active":"2023-10-19T10:38:29.592024+0000","last_peered":"2023-10-19T10:38:29.592024+0000","last_clean":"2023-10-19T10:38:29.592024+0000","last_became_active":"2023-10-19T10:38:28.594308+0000","last_became_peered":"2023-10-19T10:38:28.594308+0000","last_unstale":"2023-10-19T10:38:29.592024+0000","last_undegraded":"2023-10-19T10:38:29.592024+0000","last_fullsized":"2023-10-19T10:38:29.592024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:48:54.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592173+0000","last_change":"2023-10-19T10:38:27.589618+0000","last_active":"2023-10-19T10:38:29.592173+0000","last_peered":"2023-10-19T10:38:29.592173+0000","last_clean":"2023-10-19T10:38:29.592173+0000","last_became_active":"2023-10-19T10:38:27.589322+0000","last_became_peered":"2023-10-19T10:38:27.589322+0000","last_unstale":"2023-10-19T10:38:29.592173+0000","last_undegraded":"2023-10-19T10:38:29.592173+0000","last_fullsized":"2023-10-19T10:38:29.592173+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:13:34.146672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575636+0000","last_change":"2023-10-19T10:38:27.575636+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575636+0000","last_undegraded":"2023-10-19T10:38:27.575636+0000","last_fullsized":"2023-10-19T10:38:27.575636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937828+0000","last_change":"2023-10-19T10:38:27.584538+0000","last_active":"2023-10-19T10:38:29.937828+0000","last_peered":"2023-10-19T10:38:29.937828+0000","last_clean":"2023-10-19T10:38:29.937828+0000","last_became_active":"2023-10-19T10:38:27.584243+0000","last_became_peered":"2023-10-19T10:38:27.584243+0000","last_unstale":"2023-10-19T10:38:29.937828+0000","last_undegraded":"2023-10-19T10:38:29.937828+0000","last_fullsized":"2023-10-19T10:38:29.937828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:00:34.280277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589538+0000","last_change":"2023-10-19T10:38:28.589270+0000","last_active":"2023-10-19T10:38:29.589538+0000","last_peered":"2023-10-19T10:38:29.589538+0000","last_clean":"2023-10-19T10:38:29.589538+0000","last_became_active":"2023-10-19T10:38:28.588950+0000","last_became_peered":"2023-10-19T10:38:28.588950+0000","last_unstale":"2023-10-19T10:38:29.589538+0000","last_undegraded":"2023-10-19T10:38:29.589538+0000","last_fullsized":"2023-10-19T10:38:29.589538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:41:29.958037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589459+0000","last_change":"2023-10-19T10:38:27.591280+0000","last_active":"2023-10-19T10:38:29.589459+0000","last_peered":"2023-10-19T10:38:29.589459+0000","last_clean":"2023-10-19T10:38:29.589459+0000","last_became_active":"2023-10-19T10:38:27.590962+0000","last_became_peered":"2023-10-19T10:38:27.590962+0000","last_unstale":"2023-10-19T10:38:29.589459+0000","last_undegraded":"2023-10-19T10:38:29.589459+0000","last_fullsized":"2023-10-19T10:38:29.589459+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:00:03.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938056+0000","last_change":"2023-10-19T10:38:28.596649+0000","last_active":"2023-10-19T10:38:29.938056+0000","last_peered":"2023-10-19T10:38:29.938056+0000","last_clean":"2023-10-19T10:38:29.938056+0000","last_became_active":"2023-10-19T10:38:28.596384+0000","last_became_peered":"2023-10-19T10:38:28.596384+0000","last_unstale":"2023-10-19T10:38:29.938056+0000","last_undegraded":"2023-10-19T10:38:29.938056+0000","last_fullsized":"2023-10-19T10:38:29.938056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:32:45.922396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715110+0000","last_change":"2023-10-19T10:38:27.581103+0000","last_active":"2023-10-19T10:38:30.715110+0000","last_peered":"2023-10-19T10:38:30.715110+0000","last_clean":"2023-10-19T10:38:30.715110+0000","last_became_active":"2023-10-19T10:38:27.579968+0000","last_became_peered":"2023-10-19T10:38:27.579968+0000","last_unstale":"2023-10-19T10:38:30.715110+0000","last_undegraded":"2023-10-19T10:38:30.715110+0000","last_fullsized":"2023-10-19T10:38:30.715110+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:50:58.315710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592975+0000","last_change":"2023-10-19T10:38:28.589843+0000","last_active":"2023-10-19T10:38:29.592975+0000","last_peered":"2023-10-19T10:38:29.592975+0000","last_clean":"2023-10-19T10:38:29.592975+0000","last_became_active":"2023-10-19T10:38:28.589496+0000","last_became_peered":"2023-10-19T10:38:28.589496+0000","last_unstale":"2023-10-19T10:38:29.592975+0000","last_undegraded":"2023-10-19T10:38:29.592975+0000","last_fullsized":"2023-10-19T10:38:29.592975+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:46:28.716699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148962+0000","last_change":"2023-10-19T10:38:27.588169+0000","last_active":"2023-10-19T10:38:31.148962+0000","last_peered":"2023-10-19T10:38:31.148962+0000","last_clean":"2023-10-19T10:38:31.148962+0000","last_became_active":"2023-10-19T10:38:27.587837+0000","last_became_peered":"2023-10-19T10:38:27.587837+0000","last_unstale":"2023-10-19T10:38:31.148962+0000","last_undegraded":"2023-10-19T10:38:31.148962+0000","last_fullsized":"2023-10-19T10:38:31.148962+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:05:32.450768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593126+0000","last_change":"2023-10-19T10:38:28.593434+0000","last_active":"2023-10-19T10:38:29.593126+0000","last_peered":"2023-10-19T10:38:29.593126+0000","last_clean":"2023-10-19T10:38:29.593126+0000","last_became_active":"2023-10-19T10:38:28.593112+0000","last_became_peered":"2023-10-19T10:38:28.593112+0000","last_unstale":"2023-10-19T10:38:29.593126+0000","last_undegraded":"2023-10-19T10:38:29.593126+0000","last_fullsized":"2023-10-19T10:38:29.593126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:53:27.890189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592402+0000","last_change":"2023-10-19T10:38:27.585994+0000","last_active":"2023-10-19T10:38:29.592402+0000","last_peered":"2023-10-19T10:38:29.592402+0000","last_clean":"2023-10-19T10:38:29.592402+0000","last_became_active":"2023-10-19T10:38:27.585697+0000","last_became_peered":"2023-10-19T10:38:27.585697+0000","last_unstale":"2023-10-19T10:38:29.592402+0000","last_undegraded":"2023-10-19T10:38:29.592402+0000","last_fullsized":"2023-10-19T10:38:29.592402+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:08.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592155+0000","last_change":"2023-10-19T10:38:28.603272+0000","last_active":"2023-10-19T10:38:29.592155+0000","last_peered":"2023-10-19T10:38:29.592155+0000","last_clean":"2023-10-19T10:38:29.592155+0000","last_became_active":"2023-10-19T10:38:28.602920+0000","last_became_peered":"2023-10-19T10:38:28.602920+0000","last_unstale":"2023-10-19T10:38:29.592155+0000","last_undegraded":"2023-10-19T10:38:29.592155+0000","last_fullsized":"2023-10-19T10:38:29.592155+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:59:51.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592554+0000","last_change":"2023-10-19T10:38:27.590415+0000","last_active":"2023-10-19T10:38:29.592554+0000","last_peered":"2023-10-19T10:38:29.592554+0000","last_clean":"2023-10-19T10:38:29.592554+0000","last_became_active":"2023-10-19T10:38:27.589481+0000","last_became_peered":"2023-10-19T10:38:27.589481+0000","last_unstale":"2023-10-19T10:38:29.592554+0000","last_undegraded":"2023-10-19T10:38:29.592554+0000","last_fullsized":"2023-10-19T10:38:29.592554+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:56.479302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605441+0000","last_change":"2023-10-19T10:38:28.584626+0000","last_active":"2023-10-19T10:38:30.605441+0000","last_peered":"2023-10-19T10:38:30.605441+0000","last_clean":"2023-10-19T10:38:30.605441+0000","last_became_active":"2023-10-19T10:38:28.584346+0000","last_became_peered":"2023-10-19T10:38:28.584346+0000","last_unstale":"2023-10-19T10:38:30.605441+0000","last_undegraded":"2023-10-19T10:38:30.605441+0000","last_fullsized":"2023-10-19T10:38:30.605441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:48:30.382781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148088+0000","last_change":"2023-10-19T10:38:27.590514+0000","last_active":"2023-10-19T10:38:31.148088+0000","last_peered":"2023-10-19T10:38:31.148088+0000","last_clean":"2023-10-19T10:38:31.148088+0000","last_became_active":"2023-10-19T10:38:27.590194+0000","last_became_peered":"2023-10-19T10:38:27.590194+0000","last_unstale":"2023-10-19T10:38:31.148088+0000","last_undegraded":"2023-10-19T10:38:31.148088+0000","last_fullsized":"2023-10-19T10:38:31.148088+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:39:06.607750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589185+0000","last_change":"2023-10-19T10:38:28.590033+0000","last_active":"2023-10-19T10:38:29.589185+0000","last_peered":"2023-10-19T10:38:29.589185+0000","last_clean":"2023-10-19T10:38:29.589185+0000","last_became_active":"2023-10-19T10:38:28.589739+0000","last_became_peered":"2023-10-19T10:38:28.589739+0000","last_unstale":"2023-10-19T10:38:29.589185+0000","last_undegraded":"2023-10-19T10:38:29.589185+0000","last_fullsized":"2023-10-19T10:38:29.589185+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:50:27.417488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937527+0000","last_change":"2023-10-19T10:38:27.586164+0000","last_active":"2023-10-19T10:38:29.937527+0000","last_peered":"2023-10-19T10:38:29.937527+0000","last_clean":"2023-10-19T10:38:29.937527+0000","last_became_active":"2023-10-19T10:38:27.585838+0000","last_became_peered":"2023-10-19T10:38:27.585838+0000","last_unstale":"2023-10-19T10:38:29.937527+0000","last_undegraded":"2023-10-19T10:38:29.937527+0000","last_fullsized":"2023-10-19T10:38:29.937527+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:13:34.146672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937494+0000","last_change":"2023-10-19T10:38:28.597364+0000","last_active":"2023-10-19T10:38:29.937494+0000","last_peered":"2023-10-19T10:38:29.937494+0000","last_clean":"2023-10-19T10:38:29.937494+0000","last_became_active":"2023-10-19T10:38:28.597091+0000","last_became_peered":"2023-10-19T10:38:28.597091+0000","last_unstale":"2023-10-19T10:38:29.937494+0000","last_undegraded":"2023-10-19T10:38:29.937494+0000","last_fullsized":"2023-10-19T10:38:29.937494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:14:35.055092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.588906+0000","last_change":"2023-10-19T10:38:27.584351+0000","last_active":"2023-10-19T10:38:27.588906+0000","last_peered":"2023-10-19T10:38:27.588906+0000","last_clean":"2023-10-19T10:38:27.588906+0000","last_became_active":"2023-10-19T10:38:27.583963+0000","last_became_peered":"2023-10-19T10:38:27.583963+0000","last_unstale":"2023-10-19T10:38:27.588906+0000","last_undegraded":"2023-10-19T10:38:27.588906+0000","last_fullsized":"2023-10-19T10:38:27.588906+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:16.906120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592347+0000","last_change":"2023-10-19T10:38:28.593399+0000","last_active":"2023-10-19T10:38:29.592347+0000","last_peered":"2023-10-19T10:38:29.592347+0000","last_clean":"2023-10-19T10:38:29.592347+0000","last_became_active":"2023-10-19T10:38:28.593085+0000","last_became_peered":"2023-10-19T10:38:28.593085+0000","last_unstale":"2023-10-19T10:38:29.592347+0000","last_undegraded":"2023-10-19T10:38:29.592347+0000","last_fullsized":"2023-10-19T10:38:29.592347+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:40.058837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594279+0000","last_change":"2023-10-19T10:38:27.590766+0000","last_active":"2023-10-19T10:38:27.594279+0000","last_peered":"2023-10-19T10:38:27.594279+0000","last_clean":"2023-10-19T10:38:27.594279+0000","last_became_active":"2023-10-19T10:38:27.590488+0000","last_became_peered":"2023-10-19T10:38:27.590488+0000","last_unstale":"2023-10-19T10:38:27.594279+0000","last_undegraded":"2023-10-19T10:38:27.594279+0000","last_fullsized":"2023-10-19T10:38:27.594279+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:19:14.891257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593455+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:38:29.593455+0000","last_peered":"2023-10-19T10:38:29.593455+0000","last_clean":"2023-10-19T10:38:29.593455+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:38:29.593455+0000","last_undegraded":"2023-10-19T10:38:29.593455+0000","last_fullsized":"2023-10-19T10:38:29.593455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:07:51.595740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592306+0000","last_change":"2023-10-19T10:38:28.603225+0000","last_active":"2023-10-19T10:38:29.592306+0000","last_peered":"2023-10-19T10:38:29.592306+0000","last_clean":"2023-10-19T10:38:29.592306+0000","last_became_active":"2023-10-19T10:38:28.602887+0000","last_became_peered":"2023-10-19T10:38:28.602887+0000","last_unstale":"2023-10-19T10:38:29.592306+0000","last_undegraded":"2023-10-19T10:38:29.592306+0000","last_fullsized":"2023-10-19T10:38:29.592306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:53:14.764775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715005+0000","last_change":"2023-10-19T10:38:27.591306+0000","last_active":"2023-10-19T10:38:30.715005+0000","last_peered":"2023-10-19T10:38:30.715005+0000","last_clean":"2023-10-19T10:38:30.715005+0000","last_became_active":"2023-10-19T10:38:27.591039+0000","last_became_peered":"2023-10-19T10:38:27.591039+0000","last_unstale":"2023-10-19T10:38:30.715005+0000","last_undegraded":"2023-10-19T10:38:30.715005+0000","last_fullsized":"2023-10-19T10:38:30.715005+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:13:32.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605657+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:38:30.605657+0000","last_peered":"2023-10-19T10:38:30.605657+0000","last_clean":"2023-10-19T10:38:30.605657+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:38:30.605657+0000","last_undegraded":"2023-10-19T10:38:30.605657+0000","last_fullsized":"2023-10-19T10:38:30.605657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:43:55.690108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148046+0000","last_change":"2023-10-19T10:38:28.593159+0000","last_active":"2023-10-19T10:38:31.148046+0000","last_peered":"2023-10-19T10:38:31.148046+0000","last_clean":"2023-10-19T10:38:31.148046+0000","last_became_active":"2023-10-19T10:38:28.592848+0000","last_became_peered":"2023-10-19T10:38:28.592848+0000","last_unstale":"2023-10-19T10:38:31.148046+0000","last_undegraded":"2023-10-19T10:38:31.148046+0000","last_fullsized":"2023-10-19T10:38:31.148046+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:08:14.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605389+0000","last_change":"2023-10-19T10:38:27.586732+0000","last_active":"2023-10-19T10:38:30.605389+0000","last_peered":"2023-10-19T10:38:30.605389+0000","last_clean":"2023-10-19T10:38:30.605389+0000","last_became_active":"2023-10-19T10:38:27.586225+0000","last_became_peered":"2023-10-19T10:38:27.586225+0000","last_unstale":"2023-10-19T10:38:30.605389+0000","last_undegraded":"2023-10-19T10:38:30.605389+0000","last_fullsized":"2023-10-19T10:38:30.605389+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:45:14.987989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.578327+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:38:27.578327+0000","last_peered":"2023-10-19T10:38:27.578327+0000","last_clean":"2023-10-19T10:38:27.578327+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:38:27.578327+0000","last_undegraded":"2023-10-19T10:38:27.578327+0000","last_fullsized":"2023-10-19T10:38:27.578327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:15.047277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714772+0000","last_change":"2023-10-19T10:38:28.598900+0000","last_active":"2023-10-19T10:38:30.714772+0000","last_peered":"2023-10-19T10:38:30.714772+0000","last_clean":"2023-10-19T10:38:30.714772+0000","last_became_active":"2023-10-19T10:38:28.598635+0000","last_became_peered":"2023-10-19T10:38:28.598635+0000","last_unstale":"2023-10-19T10:38:30.714772+0000","last_undegraded":"2023-10-19T10:38:30.714772+0000","last_fullsized":"2023-10-19T10:38:30.714772+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:41:18.698468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937958+0000","last_change":"2023-10-19T10:38:27.593111+0000","last_active":"2023-10-19T10:38:29.937958+0000","last_peered":"2023-10-19T10:38:29.937958+0000","last_clean":"2023-10-19T10:38:29.937958+0000","last_became_active":"2023-10-19T10:38:27.592822+0000","last_became_peered":"2023-10-19T10:38:27.592822+0000","last_unstale":"2023-10-19T10:38:29.937958+0000","last_undegraded":"2023-10-19T10:38:29.937958+0000","last_fullsized":"2023-10-19T10:38:29.937958+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:12:20.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605407+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:38:30.605407+0000","last_peered":"2023-10-19T10:38:30.605407+0000","last_clean":"2023-10-19T10:38:30.605407+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:38:30.605407+0000","last_undegraded":"2023-10-19T10:38:30.605407+0000","last_fullsized":"2023-10-19T10:38:30.605407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:30:49.335478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593466+0000","last_change":"2023-10-19T10:38:28.587724+0000","last_active":"2023-10-19T10:38:29.593466+0000","last_peered":"2023-10-19T10:38:29.593466+0000","last_clean":"2023-10-19T10:38:29.593466+0000","last_became_active":"2023-10-19T10:38:28.587039+0000","last_became_peered":"2023-10-19T10:38:28.587039+0000","last_unstale":"2023-10-19T10:38:29.593466+0000","last_undegraded":"2023-10-19T10:38:29.593466+0000","last_fullsized":"2023-10-19T10:38:29.593466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:44:13.072808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714344+0000","last_change":"2023-10-19T10:38:27.588946+0000","last_active":"2023-10-19T10:38:30.714344+0000","last_peered":"2023-10-19T10:38:30.714344+0000","last_clean":"2023-10-19T10:38:30.714344+0000","last_became_active":"2023-10-19T10:38:27.588664+0000","last_became_peered":"2023-10-19T10:38:27.588664+0000","last_unstale":"2023-10-19T10:38:30.714344+0000","last_undegraded":"2023-10-19T10:38:30.714344+0000","last_fullsized":"2023-10-19T10:38:30.714344+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:00:34.280277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148071+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:38:31.148071+0000","last_peered":"2023-10-19T10:38:31.148071+0000","last_clean":"2023-10-19T10:38:31.148071+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:38:31.148071+0000","last_undegraded":"2023-10-19T10:38:31.148071+0000","last_fullsized":"2023-10-19T10:38:31.148071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:31:07.514042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937705+0000","last_change":"2023-10-19T10:38:28.589427+0000","last_active":"2023-10-19T10:38:29.937705+0000","last_peered":"2023-10-19T10:38:29.937705+0000","last_clean":"2023-10-19T10:38:29.937705+0000","last_became_active":"2023-10-19T10:38:28.588927+0000","last_became_peered":"2023-10-19T10:38:28.588927+0000","last_unstale":"2023-10-19T10:38:29.937705+0000","last_undegraded":"2023-10-19T10:38:29.937705+0000","last_fullsized":"2023-10-19T10:38:29.937705+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:09.216870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.594316+0000","last_change":"2023-10-19T10:38:27.590779+0000","last_active":"2023-10-19T10:38:27.594316+0000","last_peered":"2023-10-19T10:38:27.594316+0000","last_clean":"2023-10-19T10:38:27.594316+0000","last_became_active":"2023-10-19T10:38:27.590482+0000","last_became_peered":"2023-10-19T10:38:27.590482+0000","last_unstale":"2023-10-19T10:38:27.594316+0000","last_undegraded":"2023-10-19T10:38:27.594316+0000","last_fullsized":"2023-10-19T10:38:27.594316+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:10:42.478900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589099+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:38:29.589099+0000","last_peered":"2023-10-19T10:38:29.589099+0000","last_clean":"2023-10-19T10:38:29.589099+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:38:29.589099+0000","last_undegraded":"2023-10-19T10:38:29.589099+0000","last_fullsized":"2023-10-19T10:38:29.589099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:06.119282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2023-10-19T10:38:27.575245+0000","last_change":"2023-10-19T10:38:27.575245+0000","last_active":"2023-10-19T10:38:27.561034+0000","last_peered":"2023-10-19T10:38:27.561034+0000","last_clean":"2023-10-19T10:38:27.561034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-19T10:38:27.575245+0000","last_undegraded":"2023-10-19T10:38:27.575245+0000","last_fullsized":"2023-10-19T10:38:27.575245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605422+0000","last_change":"2023-10-19T10:38:27.584088+0000","last_active":"2023-10-19T10:38:30.605422+0000","last_peered":"2023-10-19T10:38:30.605422+0000","last_clean":"2023-10-19T10:38:30.605422+0000","last_became_active":"2023-10-19T10:38:27.583803+0000","last_became_peered":"2023-10-19T10:38:27.583803+0000","last_unstale":"2023-10-19T10:38:30.605422+0000","last_undegraded":"2023-10-19T10:38:30.605422+0000","last_fullsized":"2023-10-19T10:38:30.605422+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:16.906120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.576832+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:38:27.576832+0000","last_peered":"2023-10-19T10:38:27.576832+0000","last_clean":"2023-10-19T10:38:27.576832+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:38:27.576832+0000","last_undegraded":"2023-10-19T10:38:27.576832+0000","last_fullsized":"2023-10-19T10:38:27.576832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:43:37.587705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938027+0000","last_change":"2023-10-19T10:38:28.589570+0000","last_active":"2023-10-19T10:38:29.938027+0000","last_peered":"2023-10-19T10:38:29.938027+0000","last_clean":"2023-10-19T10:38:29.938027+0000","last_became_active":"2023-10-19T10:38:28.589288+0000","last_became_peered":"2023-10-19T10:38:28.589288+0000","last_unstale":"2023-10-19T10:38:29.938027+0000","last_undegraded":"2023-10-19T10:38:29.938027+0000","last_fullsized":"2023-10-19T10:38:29.938027+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:39:07.615819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.149113+0000","last_change":"2023-10-19T10:38:27.591386+0000","last_active":"2023-10-19T10:38:31.149113+0000","last_peered":"2023-10-19T10:38:31.149113+0000","last_clean":"2023-10-19T10:38:31.149113+0000","last_became_active":"2023-10-19T10:38:27.591057+0000","last_became_peered":"2023-10-19T10:38:27.591057+0000","last_unstale":"2023-10-19T10:38:31.149113+0000","last_undegraded":"2023-10-19T10:38:31.149113+0000","last_fullsized":"2023-10-19T10:38:31.149113+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:47:00.210895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148284+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:38:31.148284+0000","last_peered":"2023-10-19T10:38:31.148284+0000","last_clean":"2023-10-19T10:38:31.148284+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:38:31.148284+0000","last_undegraded":"2023-10-19T10:38:31.148284+0000","last_fullsized":"2023-10-19T10:38:31.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:07.696839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-19T10:38:27.577404+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:38:27.577404+0000","last_peered":"2023-10-19T10:38:27.577404+0000","last_clean":"2023-10-19T10:38:27.577404+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:38:27.577404+0000","last_undegraded":"2023-10-19T10:38:27.577404+0000","last_fullsized":"2023-10-19T10:38:27.577404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:45.342865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":794443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.501}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.377},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.438}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":495616,"data_stored":219168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.627},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":211593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.376}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.626},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.495},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.42799999999999999}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":213135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.627},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":499712,"data_stored":219134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":503808,"data_stored":220130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.308},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.628}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":491520,"data_stored":218566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.624},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.38}]},{"osd":3,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.752}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1085440,"data_stored":808898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.377},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.63},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:38:36.057 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:38:36.179 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.177+0000 7fa3356f9640 1 Processor -- start 2023-10-19T10:38:36.180 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.177+0000 7fa3356f9640 1 -- start start 2023-10-19T10:38:36.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.177+0000 7fa3356f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa330159280 0x7fa33015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.177+0000 7fa3356f9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33015d2d0 con 0x7fa330159e90 2023-10-19T10:38:36.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.177+0000 7fa3356f9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300ca480 con 0x7fa3300cd7d0 2023-10-19T10:38:36.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.177+0000 7fa3356f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa330159990 con 0x7fa330159280 2023-10-19T10:38:36.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa330159e90 0x7fa33006b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58908/0 (socket says 172.21.15.50:58908) 2023-10-19T10:38:36.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32f7fe640 1 -- 172.21.15.50:0/3785760591 learned_addr learned my addr 172.21.15.50:0/3785760591 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:36.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32effd640 1 --2- 172.21.15.50:0/3785760591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa330159280 0x7fa33015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589707972 0 0) 0x7fa33015d2d0 con 0x7fa330159e90 2023-10-19T10:38:36.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa318003520 con 0x7fa330159e90 2023-10-19T10:38:36.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684087586 0 0) 0x7fa3300ca480 con 0x7fa3300cd7d0 2023-10-19T10:38:36.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa33015d2d0 con 0x7fa3300cd7d0 2023-10-19T10:38:36.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1656948379 0 0) 0x7fa318003520 con 0x7fa330159e90 2023-10-19T10:38:36.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32effd640 1 -- 172.21.15.50:0/3785760591 >> v1:172.21.15.183:6789/0 conn(0x7fa3300cd7d0 legacy=0x7fa3300c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32effd640 1 -- 172.21.15.50:0/3785760591 >> v1:172.21.15.50:6789/0 conn(0x7fa330159e90 legacy=0x7fa33006b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.178+0000 7fa32effd640 1 -- 172.21.15.50:0/3785760591 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33015d500 con 0x7fa330159280 2023-10-19T10:38:36.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.179+0000 7fa32effd640 1 --2- 172.21.15.50:0/3785760591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa330159280 0x7fa33015cd90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa3300ca670 tx=0x7fa32002b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4f037c67a11dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.179+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3280019f0 con 0x7fa330159280 2023-10-19T10:38:36.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.179+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200314d0 con 0x7fa330159280 2023-10-19T10:38:36.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.179+0000 7fa32dffb640 1 -- 172.21.15.50:0/3785760591 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320031860 con 0x7fa330159280 2023-10-19T10:38:36.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 -- 172.21.15.50:0/3785760591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa330159280 msgr2=0x7fa33015cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3785760591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa330159280 0x7fa33015cd90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa3300ca670 tx=0x7fa32002b7f0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 -- 172.21.15.50:0/3785760591 shutdown_connections 2023-10-19T10:38:36.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3785760591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa330159280 0x7fa33015cd90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 -- 172.21.15.50:0/3785760591 >> 172.21.15.50:0/3785760591 conn(0x7fa330069e90 msgr2=0x7fa3300afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:36.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 -- 172.21.15.50:0/3785760591 shutdown_connections 2023-10-19T10:38:36.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.180+0000 7fa3356f9640 1 -- 172.21.15.50:0/3785760591 wait complete. 2023-10-19T10:38:36.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.181+0000 7fa3356f9640 1 Processor -- start 2023-10-19T10:38:36.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.181+0000 7fa3356f9640 1 -- start start 2023-10-19T10:38:36.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa3356f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3300cd7d0 0x7fa3300b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa3356f9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 0x7fa3300cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa3356f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa330159e90 0x7fa3300c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa3356f9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33014c920 con 0x7fa330159280 2023-10-19T10:38:36.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa3356f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300c2d10 con 0x7fa330159e90 2023-10-19T10:38:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa3356f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300c2100 con 0x7fa3300cd7d0 2023-10-19T10:38:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa32e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 0x7fa3300cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa32e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 0x7fa3300cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41998/0 (socket says 172.21.15.50:41998) 2023-10-19T10:38:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa32e7fc640 1 -- 172.21.15.50:0/3349561650 learned_addr learned my addr 172.21.15.50:0/3349561650 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.182+0000 7fa32effd640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3300cd7d0 0x7fa3300b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32f7fe640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa330159e90 0x7fa3300c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32e7fc640 1 -- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3300cd7d0 msgr2=0x7fa3300b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32e7fc640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3300cd7d0 0x7fa3300b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32e7fc640 1 -- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa330159e90 msgr2=0x7fa3300c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32e7fc640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa330159e90 0x7fa3300c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32e7fc640 1 -- 172.21.15.50:0/3349561650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009400 con 0x7fa330159280 2023-10-19T10:38:36.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32effd640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3300cd7d0 0x7fa3300b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:36.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa32e7fc640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 0x7fa3300cb510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c490 tx=0x7fa32400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3240052d0 con 0x7fa330159280 2023-10-19T10:38:36.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa324005430 con 0x7fa330159280 2023-10-19T10:38:36.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3300c23e0 con 0x7fa330159280 2023-10-19T10:38:36.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324012870 con 0x7fa330159280 2023-10-19T10:38:36.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.183+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3300c29a0 con 0x7fa330159280 2023-10-19T10:38:36.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.184+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa330073990 con 0x7fa330159280 2023-10-19T10:38:36.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.185+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3240129d0 con 0x7fa330159280 2023-10-19T10:38:36.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.185+0000 7fa3177fe640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa31c06a050 0x7fa31c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.185+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa324088270 con 0x7fa330159280 2023-10-19T10:38:36.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.185+0000 7fa32effd640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa31c06a050 0x7fa31c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.186+0000 7fa32effd640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa31c06a050 0x7fa31c06c510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa320036000 tx=0x7fa320008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.189+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa324057300 con 0x7fa330159280 2023-10-19T10:38:36.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.362+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa33015dac0 con 0x7fa31c06a050 2023-10-19T10:38:36.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.370+0000 7fa3177fe640 1 -- 172.21.15.50:0/3349561650 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311684 (secure 0 0 0) 0x7fa320044090 con 0x7fa31c06a050 2023-10-19T10:38:36.376 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:36.378 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:38:36.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.373+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa31c06a050 msgr2=0x7fa31c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.373+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa31c06a050 0x7fa31c06c510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa320036000 tx=0x7fa320008030 comp rx=0 tx=0).stop 2023-10-19T10:38:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.373+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 msgr2=0x7fa3300cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.373+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 0x7fa3300cb510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c490 tx=0x7fa32400c950 comp rx=0 tx=0).stop 2023-10-19T10:38:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 shutdown_connections 2023-10-19T10:38:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa31c06a050 0x7fa31c06c510 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa330159e90 0x7fa3300c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330159280 0x7fa3300cb510 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 --2- 172.21.15.50:0/3349561650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3300cd7d0 0x7fa3300b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 >> 172.21.15.50:0/3349561650 conn(0x7fa330069e90 msgr2=0x7fa3300aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:36.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 shutdown_connections 2023-10-19T10:38:36.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.374+0000 7fa3356f9640 1 -- 172.21.15.50:0/3349561650 wait complete. 2023-10-19T10:38:36.387 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-10-19T10:38:36.118723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237572,"kb_used_data":5008,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737148,"statfs":{"total":773094113280,"available":772850839552,"internally_reserved":0,"allocated":5128192,"data_stored":2919578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.629650"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148322+0000","last_change":"2023-10-19T10:38:28.587086+0000","last_active":"2023-10-19T10:38:31.148322+0000","last_peered":"2023-10-19T10:38:31.148322+0000","last_clean":"2023-10-19T10:38:31.148322+0000","last_became_active":"2023-10-19T10:38:28.586439+0000","last_became_peered":"2023-10-19T10:38:28.586439+0000","last_unstale":"2023-10-19T10:38:31.148322+0000","last_undegraded":"2023-10-19T10:38:31.148322+0000","last_fullsized":"2023-10-19T10:38:31.148322+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:08:14.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148738+0000","last_change":"2023-10-19T10:38:28.590368+0000","last_active":"2023-10-19T10:38:31.148738+0000","last_peered":"2023-10-19T10:38:31.148738+0000","last_clean":"2023-10-19T10:38:31.148738+0000","last_became_active":"2023-10-19T10:38:28.590116+0000","last_became_peered":"2023-10-19T10:38:28.590116+0000","last_unstale":"2023-10-19T10:38:31.148738+0000","last_undegraded":"2023-10-19T10:38:31.148738+0000","last_fullsized":"2023-10-19T10:38:31.148738+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:15:53.169196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714817+0000","last_change":"2023-10-19T10:38:28.596684+0000","last_active":"2023-10-19T10:38:30.714817+0000","last_peered":"2023-10-19T10:38:30.714817+0000","last_clean":"2023-10-19T10:38:30.714817+0000","last_became_active":"2023-10-19T10:38:28.596346+0000","last_became_peered":"2023-10-19T10:38:28.596346+0000","last_unstale":"2023-10-19T10:38:30.714817+0000","last_undegraded":"2023-10-19T10:38:30.714817+0000","last_fullsized":"2023-10-19T10:38:30.714817+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:15:12.478642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937673+0000","last_change":"2023-10-19T10:38:28.586191+0000","last_active":"2023-10-19T10:38:29.937673+0000","last_peered":"2023-10-19T10:38:29.937673+0000","last_clean":"2023-10-19T10:38:29.937673+0000","last_became_active":"2023-10-19T10:38:28.585819+0000","last_became_peered":"2023-10-19T10:38:28.585819+0000","last_unstale":"2023-10-19T10:38:29.937673+0000","last_undegraded":"2023-10-19T10:38:29.937673+0000","last_fullsized":"2023-10-19T10:38:29.937673+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:40.058837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148191+0000","last_change":"2023-10-19T10:38:28.587084+0000","last_active":"2023-10-19T10:38:31.148191+0000","last_peered":"2023-10-19T10:38:31.148191+0000","last_clean":"2023-10-19T10:38:31.148191+0000","last_became_active":"2023-10-19T10:38:28.586717+0000","last_became_peered":"2023-10-19T10:38:28.586717+0000","last_unstale":"2023-10-19T10:38:31.148191+0000","last_undegraded":"2023-10-19T10:38:31.148191+0000","last_fullsized":"2023-10-19T10:38:31.148191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:35:04.830904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592127+0000","last_change":"2023-10-19T10:38:28.591287+0000","last_active":"2023-10-19T10:38:29.592127+0000","last_peered":"2023-10-19T10:38:29.592127+0000","last_clean":"2023-10-19T10:38:29.592127+0000","last_became_active":"2023-10-19T10:38:28.590992+0000","last_became_peered":"2023-10-19T10:38:28.590992+0000","last_unstale":"2023-10-19T10:38:29.592127+0000","last_undegraded":"2023-10-19T10:38:29.592127+0000","last_fullsized":"2023-10-19T10:38:29.592127+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:14:35.055092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148679+0000","last_change":"2023-10-19T10:38:28.581694+0000","last_active":"2023-10-19T10:38:31.148679+0000","last_peered":"2023-10-19T10:38:31.148679+0000","last_clean":"2023-10-19T10:38:31.148679+0000","last_became_active":"2023-10-19T10:38:28.581423+0000","last_became_peered":"2023-10-19T10:38:28.581423+0000","last_unstale":"2023-10-19T10:38:31.148679+0000","last_undegraded":"2023-10-19T10:38:31.148679+0000","last_fullsized":"2023-10-19T10:38:31.148679+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:56:16.974974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937431+0000","last_change":"2023-10-19T10:38:28.592196+0000","last_active":"2023-10-19T10:38:29.937431+0000","last_peered":"2023-10-19T10:38:29.937431+0000","last_clean":"2023-10-19T10:38:29.937431+0000","last_became_active":"2023-10-19T10:38:28.591908+0000","last_became_peered":"2023-10-19T10:38:28.591908+0000","last_unstale":"2023-10-19T10:38:29.937431+0000","last_undegraded":"2023-10-19T10:38:29.937431+0000","last_fullsized":"2023-10-19T10:38:29.937431+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:40:46.629010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938007+0000","last_change":"2023-10-19T10:38:28.607434+0000","last_active":"2023-10-19T10:38:29.938007+0000","last_peered":"2023-10-19T10:38:29.938007+0000","last_clean":"2023-10-19T10:38:29.938007+0000","last_became_active":"2023-10-19T10:38:28.607177+0000","last_became_peered":"2023-10-19T10:38:28.607177+0000","last_unstale":"2023-10-19T10:38:29.938007+0000","last_undegraded":"2023-10-19T10:38:29.938007+0000","last_fullsized":"2023-10-19T10:38:29.938007+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:57.487371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938012+0000","last_change":"2023-10-19T10:38:28.586306+0000","last_active":"2023-10-19T10:38:29.938012+0000","last_peered":"2023-10-19T10:38:29.938012+0000","last_clean":"2023-10-19T10:38:29.938012+0000","last_became_active":"2023-10-19T10:38:28.585875+0000","last_became_peered":"2023-10-19T10:38:28.585875+0000","last_unstale":"2023-10-19T10:38:29.938012+0000","last_undegraded":"2023-10-19T10:38:29.938012+0000","last_fullsized":"2023-10-19T10:38:29.938012+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:08:14.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937798+0000","last_change":"2023-10-19T10:38:28.607171+0000","last_active":"2023-10-19T10:38:29.937798+0000","last_peered":"2023-10-19T10:38:29.937798+0000","last_clean":"2023-10-19T10:38:29.937798+0000","last_became_active":"2023-10-19T10:38:28.606827+0000","last_became_peered":"2023-10-19T10:38:28.606827+0000","last_unstale":"2023-10-19T10:38:29.937798+0000","last_undegraded":"2023-10-19T10:38:29.937798+0000","last_fullsized":"2023-10-19T10:38:29.937798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:24.307807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148918+0000","last_change":"2023-10-19T10:38:28.591464+0000","last_active":"2023-10-19T10:38:31.148918+0000","last_peered":"2023-10-19T10:38:31.148918+0000","last_clean":"2023-10-19T10:38:31.148918+0000","last_became_active":"2023-10-19T10:38:28.591209+0000","last_became_peered":"2023-10-19T10:38:28.591209+0000","last_unstale":"2023-10-19T10:38:31.148918+0000","last_undegraded":"2023-10-19T10:38:31.148918+0000","last_fullsized":"2023-10-19T10:38:31.148918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:48:09.095111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589513+0000","last_change":"2023-10-19T10:38:28.601166+0000","last_active":"2023-10-19T10:38:29.589513+0000","last_peered":"2023-10-19T10:38:29.589513+0000","last_clean":"2023-10-19T10:38:29.589513+0000","last_became_active":"2023-10-19T10:38:28.600897+0000","last_became_peered":"2023-10-19T10:38:28.600897+0000","last_unstale":"2023-10-19T10:38:29.589513+0000","last_undegraded":"2023-10-19T10:38:29.589513+0000","last_fullsized":"2023-10-19T10:38:29.589513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:46:28.716699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937573+0000","last_change":"2023-10-19T10:38:28.597332+0000","last_active":"2023-10-19T10:38:29.937573+0000","last_peered":"2023-10-19T10:38:29.937573+0000","last_clean":"2023-10-19T10:38:29.937573+0000","last_became_active":"2023-10-19T10:38:28.597050+0000","last_became_peered":"2023-10-19T10:38:28.597050+0000","last_unstale":"2023-10-19T10:38:29.937573+0000","last_undegraded":"2023-10-19T10:38:29.937573+0000","last_fullsized":"2023-10-19T10:38:29.937573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:35:51.161014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592024+0000","last_change":"2023-10-19T10:38:28.588702+0000","last_active":"2023-10-19T10:38:29.592024+0000","last_peered":"2023-10-19T10:38:29.592024+0000","last_clean":"2023-10-19T10:38:29.592024+0000","last_became_active":"2023-10-19T10:38:28.588042+0000","last_became_peered":"2023-10-19T10:38:28.588042+0000","last_unstale":"2023-10-19T10:38:29.592024+0000","last_undegraded":"2023-10-19T10:38:29.592024+0000","last_fullsized":"2023-10-19T10:38:29.592024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:40:46.629010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592702+0000","last_change":"2023-10-19T10:38:28.592184+0000","last_active":"2023-10-19T10:38:29.592702+0000","last_peered":"2023-10-19T10:38:29.592702+0000","last_clean":"2023-10-19T10:38:29.592702+0000","last_became_active":"2023-10-19T10:38:28.591679+0000","last_became_peered":"2023-10-19T10:38:28.591679+0000","last_unstale":"2023-10-19T10:38:29.592702+0000","last_undegraded":"2023-10-19T10:38:29.592702+0000","last_fullsized":"2023-10-19T10:38:29.592702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:20:35.229068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715047+0000","last_change":"2023-10-19T10:38:28.588006+0000","last_active":"2023-10-19T10:38:30.715047+0000","last_peered":"2023-10-19T10:38:30.715047+0000","last_clean":"2023-10-19T10:38:30.715047+0000","last_became_active":"2023-10-19T10:38:28.587712+0000","last_became_peered":"2023-10-19T10:38:28.587712+0000","last_unstale":"2023-10-19T10:38:30.715047+0000","last_undegraded":"2023-10-19T10:38:30.715047+0000","last_fullsized":"2023-10-19T10:38:30.715047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:26:35.409939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714954+0000","last_change":"2023-10-19T10:38:28.601329+0000","last_active":"2023-10-19T10:38:30.714954+0000","last_peered":"2023-10-19T10:38:30.714954+0000","last_clean":"2023-10-19T10:38:30.714954+0000","last_became_active":"2023-10-19T10:38:28.600859+0000","last_became_peered":"2023-10-19T10:38:28.600859+0000","last_unstale":"2023-10-19T10:38:30.714954+0000","last_undegraded":"2023-10-19T10:38:30.714954+0000","last_fullsized":"2023-10-19T10:38:30.714954+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:16:56.445788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593221+0000","last_change":"2023-10-19T10:38:28.581733+0000","last_active":"2023-10-19T10:38:29.593221+0000","last_peered":"2023-10-19T10:38:29.593221+0000","last_clean":"2023-10-19T10:38:29.593221+0000","last_became_active":"2023-10-19T10:38:28.581421+0000","last_became_peered":"2023-10-19T10:38:28.581421+0000","last_unstale":"2023-10-19T10:38:29.593221+0000","last_undegraded":"2023-10-19T10:38:29.593221+0000","last_fullsized":"2023-10-19T10:38:29.593221+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:58:04.017833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589407+0000","last_change":"2023-10-19T10:38:28.586586+0000","last_active":"2023-10-19T10:38:29.589407+0000","last_peered":"2023-10-19T10:38:29.589407+0000","last_clean":"2023-10-19T10:38:29.589407+0000","last_became_active":"2023-10-19T10:38:28.586288+0000","last_became_peered":"2023-10-19T10:38:28.586288+0000","last_unstale":"2023-10-19T10:38:29.589407+0000","last_undegraded":"2023-10-19T10:38:29.589407+0000","last_fullsized":"2023-10-19T10:38:29.589407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:52:51.318834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592275+0000","last_change":"2023-10-19T10:38:28.601706+0000","last_active":"2023-10-19T10:38:29.592275+0000","last_peered":"2023-10-19T10:38:29.592275+0000","last_clean":"2023-10-19T10:38:29.592275+0000","last_became_active":"2023-10-19T10:38:28.601355+0000","last_became_peered":"2023-10-19T10:38:28.601355+0000","last_unstale":"2023-10-19T10:38:29.592275+0000","last_undegraded":"2023-10-19T10:38:29.592275+0000","last_fullsized":"2023-10-19T10:38:29.592275+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:35:51.161014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937711+0000","last_change":"2023-10-19T10:38:28.586233+0000","last_active":"2023-10-19T10:38:29.937711+0000","last_peered":"2023-10-19T10:38:29.937711+0000","last_clean":"2023-10-19T10:38:29.937711+0000","last_became_active":"2023-10-19T10:38:28.585930+0000","last_became_peered":"2023-10-19T10:38:28.585930+0000","last_unstale":"2023-10-19T10:38:29.937711+0000","last_undegraded":"2023-10-19T10:38:29.937711+0000","last_fullsized":"2023-10-19T10:38:29.937711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:48:53.289286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592726+0000","last_change":"2023-10-19T10:38:28.584879+0000","last_active":"2023-10-19T10:38:29.592726+0000","last_peered":"2023-10-19T10:38:29.592726+0000","last_clean":"2023-10-19T10:38:29.592726+0000","last_became_active":"2023-10-19T10:38:28.584623+0000","last_became_peered":"2023-10-19T10:38:28.584623+0000","last_unstale":"2023-10-19T10:38:29.592726+0000","last_undegraded":"2023-10-19T10:38:29.592726+0000","last_fullsized":"2023-10-19T10:38:29.592726+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:00:04.990551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593377+0000","last_change":"2023-10-19T10:38:28.586155+0000","last_active":"2023-10-19T10:38:29.593377+0000","last_peered":"2023-10-19T10:38:29.593377+0000","last_clean":"2023-10-19T10:38:29.593377+0000","last_became_active":"2023-10-19T10:38:28.585600+0000","last_became_peered":"2023-10-19T10:38:28.585600+0000","last_unstale":"2023-10-19T10:38:29.593377+0000","last_undegraded":"2023-10-19T10:38:29.593377+0000","last_fullsized":"2023-10-19T10:38:29.593377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:07:38.741917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593179+0000","last_change":"2023-10-19T10:38:28.600175+0000","last_active":"2023-10-19T10:38:29.593179+0000","last_peered":"2023-10-19T10:38:29.593179+0000","last_clean":"2023-10-19T10:38:29.593179+0000","last_became_active":"2023-10-19T10:38:28.599704+0000","last_became_peered":"2023-10-19T10:38:28.599704+0000","last_unstale":"2023-10-19T10:38:29.593179+0000","last_undegraded":"2023-10-19T10:38:29.593179+0000","last_fullsized":"2023-10-19T10:38:29.593179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:22:55.924502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589544+0000","last_change":"2023-10-19T10:38:28.589160+0000","last_active":"2023-10-19T10:38:29.589544+0000","last_peered":"2023-10-19T10:38:29.589544+0000","last_clean":"2023-10-19T10:38:29.589544+0000","last_became_active":"2023-10-19T10:38:28.588852+0000","last_became_peered":"2023-10-19T10:38:28.588852+0000","last_unstale":"2023-10-19T10:38:29.589544+0000","last_undegraded":"2023-10-19T10:38:29.589544+0000","last_fullsized":"2023-10-19T10:38:29.589544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:53:27.890189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714933+0000","last_change":"2023-10-19T10:38:28.587517+0000","last_active":"2023-10-19T10:38:30.714933+0000","last_peered":"2023-10-19T10:38:30.714933+0000","last_clean":"2023-10-19T10:38:30.714933+0000","last_became_active":"2023-10-19T10:38:28.587261+0000","last_became_peered":"2023-10-19T10:38:28.587261+0000","last_unstale":"2023-10-19T10:38:30.714933+0000","last_undegraded":"2023-10-19T10:38:30.714933+0000","last_fullsized":"2023-10-19T10:38:30.714933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:51.984473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148803+0000","last_change":"2023-10-19T10:38:28.602166+0000","last_active":"2023-10-19T10:38:31.148803+0000","last_peered":"2023-10-19T10:38:31.148803+0000","last_clean":"2023-10-19T10:38:31.148803+0000","last_became_active":"2023-10-19T10:38:28.601867+0000","last_became_peered":"2023-10-19T10:38:28.601867+0000","last_unstale":"2023-10-19T10:38:31.148803+0000","last_undegraded":"2023-10-19T10:38:31.148803+0000","last_fullsized":"2023-10-19T10:38:31.148803+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:56:16.974974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593093+0000","last_change":"2023-10-19T10:38:28.593225+0000","last_active":"2023-10-19T10:38:29.593093+0000","last_peered":"2023-10-19T10:38:29.593093+0000","last_clean":"2023-10-19T10:38:29.593093+0000","last_became_active":"2023-10-19T10:38:28.592959+0000","last_became_peered":"2023-10-19T10:38:28.592959+0000","last_unstale":"2023-10-19T10:38:29.593093+0000","last_undegraded":"2023-10-19T10:38:29.593093+0000","last_fullsized":"2023-10-19T10:38:29.593093+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:41:29.958037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714820+0000","last_change":"2023-10-19T10:38:28.588475+0000","last_active":"2023-10-19T10:38:30.714820+0000","last_peered":"2023-10-19T10:38:30.714820+0000","last_clean":"2023-10-19T10:38:30.714820+0000","last_became_active":"2023-10-19T10:38:28.588181+0000","last_became_peered":"2023-10-19T10:38:28.588181+0000","last_unstale":"2023-10-19T10:38:30.714820+0000","last_undegraded":"2023-10-19T10:38:30.714820+0000","last_fullsized":"2023-10-19T10:38:30.714820+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:47:09.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592226+0000","last_change":"2023-10-19T10:38:28.588873+0000","last_active":"2023-10-19T10:38:29.592226+0000","last_peered":"2023-10-19T10:38:29.592226+0000","last_clean":"2023-10-19T10:38:29.592226+0000","last_became_active":"2023-10-19T10:38:28.588411+0000","last_became_peered":"2023-10-19T10:38:28.588411+0000","last_unstale":"2023-10-19T10:38:29.592226+0000","last_undegraded":"2023-10-19T10:38:29.592226+0000","last_fullsized":"2023-10-19T10:38:29.592226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:25:06.197277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589140+0000","last_change":"2023-10-19T10:38:28.597732+0000","last_active":"2023-10-19T10:38:29.589140+0000","last_peered":"2023-10-19T10:38:29.589140+0000","last_clean":"2023-10-19T10:38:29.589140+0000","last_became_active":"2023-10-19T10:38:28.597472+0000","last_became_peered":"2023-10-19T10:38:28.597472+0000","last_unstale":"2023-10-19T10:38:29.589140+0000","last_undegraded":"2023-10-19T10:38:29.589140+0000","last_fullsized":"2023-10-19T10:38:29.589140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:06:10.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605630+0000","last_change":"2023-10-19T10:38:28.589568+0000","last_active":"2023-10-19T10:38:30.605630+0000","last_peered":"2023-10-19T10:38:30.605630+0000","last_clean":"2023-10-19T10:38:30.605630+0000","last_became_active":"2023-10-19T10:38:28.589294+0000","last_became_peered":"2023-10-19T10:38:28.589294+0000","last_unstale":"2023-10-19T10:38:30.605630+0000","last_undegraded":"2023-10-19T10:38:30.605630+0000","last_fullsized":"2023-10-19T10:38:30.605630+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:19:15.899326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592527+0000","last_change":"2023-10-19T10:38:27.590076+0000","last_active":"2023-10-19T10:38:29.592527+0000","last_peered":"2023-10-19T10:38:29.592527+0000","last_clean":"2023-10-19T10:38:29.592527+0000","last_became_active":"2023-10-19T10:38:27.589685+0000","last_became_peered":"2023-10-19T10:38:27.589685+0000","last_unstale":"2023-10-19T10:38:29.592527+0000","last_undegraded":"2023-10-19T10:38:29.592527+0000","last_fullsized":"2023-10-19T10:38:29.592527+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:12:20.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593371+0000","last_change":"2023-10-19T10:38:28.586452+0000","last_active":"2023-10-19T10:38:29.593371+0000","last_peered":"2023-10-19T10:38:29.593371+0000","last_clean":"2023-10-19T10:38:29.593371+0000","last_became_active":"2023-10-19T10:38:28.585845+0000","last_became_peered":"2023-10-19T10:38:28.585845+0000","last_unstale":"2023-10-19T10:38:29.593371+0000","last_undegraded":"2023-10-19T10:38:29.593371+0000","last_fullsized":"2023-10-19T10:38:29.593371+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:06:13.597070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715110+0000","last_change":"2023-10-19T10:38:27.583864+0000","last_active":"2023-10-19T10:38:30.715110+0000","last_peered":"2023-10-19T10:38:30.715110+0000","last_clean":"2023-10-19T10:38:30.715110+0000","last_became_active":"2023-10-19T10:38:27.583595+0000","last_became_peered":"2023-10-19T10:38:27.583595+0000","last_unstale":"2023-10-19T10:38:30.715110+0000","last_undegraded":"2023-10-19T10:38:30.715110+0000","last_fullsized":"2023-10-19T10:38:30.715110+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:48:27.881884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148955+0000","last_change":"2023-10-19T10:38:28.596911+0000","last_active":"2023-10-19T10:38:31.148955+0000","last_peered":"2023-10-19T10:38:31.148955+0000","last_clean":"2023-10-19T10:38:31.148955+0000","last_became_active":"2023-10-19T10:38:28.596560+0000","last_became_peered":"2023-10-19T10:38:28.596560+0000","last_unstale":"2023-10-19T10:38:31.148955+0000","last_undegraded":"2023-10-19T10:38:31.148955+0000","last_fullsized":"2023-10-19T10:38:31.148955+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:44:42.869570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605742+0000","last_change":"2023-10-19T10:38:27.585185+0000","last_active":"2023-10-19T10:38:30.605742+0000","last_peered":"2023-10-19T10:38:30.605742+0000","last_clean":"2023-10-19T10:38:30.605742+0000","last_became_active":"2023-10-19T10:38:27.584912+0000","last_became_peered":"2023-10-19T10:38:27.584912+0000","last_unstale":"2023-10-19T10:38:30.605742+0000","last_undegraded":"2023-10-19T10:38:30.605742+0000","last_fullsized":"2023-10-19T10:38:30.605742+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:17:13.906630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605781+0000","last_change":"2023-10-19T10:38:28.582163+0000","last_active":"2023-10-19T10:38:30.605781+0000","last_peered":"2023-10-19T10:38:30.605781+0000","last_clean":"2023-10-19T10:38:30.605781+0000","last_became_active":"2023-10-19T10:38:28.581862+0000","last_became_peered":"2023-10-19T10:38:28.581862+0000","last_unstale":"2023-10-19T10:38:30.605781+0000","last_undegraded":"2023-10-19T10:38:30.605781+0000","last_fullsized":"2023-10-19T10:38:30.605781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:10:43.486969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148146+0000","last_change":"2023-10-19T10:38:27.584383+0000","last_active":"2023-10-19T10:38:31.148146+0000","last_peered":"2023-10-19T10:38:31.148146+0000","last_clean":"2023-10-19T10:38:31.148146+0000","last_became_active":"2023-10-19T10:38:27.583945+0000","last_became_peered":"2023-10-19T10:38:27.583945+0000","last_unstale":"2023-10-19T10:38:31.148146+0000","last_undegraded":"2023-10-19T10:38:31.148146+0000","last_fullsized":"2023-10-19T10:38:31.148146+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:12:20.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714814+0000","last_change":"2023-10-19T10:38:28.590904+0000","last_active":"2023-10-19T10:38:30.714814+0000","last_peered":"2023-10-19T10:38:30.714814+0000","last_clean":"2023-10-19T10:38:30.714814+0000","last_became_active":"2023-10-19T10:38:28.590604+0000","last_became_peered":"2023-10-19T10:38:28.590604+0000","last_unstale":"2023-10-19T10:38:30.714814+0000","last_undegraded":"2023-10-19T10:38:30.714814+0000","last_fullsized":"2023-10-19T10:38:30.714814+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:18:22.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592407+0000","last_change":"2023-10-19T10:38:27.589945+0000","last_active":"2023-10-19T10:38:29.592407+0000","last_peered":"2023-10-19T10:38:29.592407+0000","last_clean":"2023-10-19T10:38:29.592407+0000","last_became_active":"2023-10-19T10:38:27.589196+0000","last_became_peered":"2023-10-19T10:38:27.589196+0000","last_unstale":"2023-10-19T10:38:29.592407+0000","last_undegraded":"2023-10-19T10:38:29.592407+0000","last_fullsized":"2023-10-19T10:38:29.592407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:01:23.299738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937558+0000","last_change":"2023-10-19T10:38:28.609017+0000","last_active":"2023-10-19T10:38:29.937558+0000","last_peered":"2023-10-19T10:38:29.937558+0000","last_clean":"2023-10-19T10:38:29.937558+0000","last_became_active":"2023-10-19T10:38:28.608742+0000","last_became_peered":"2023-10-19T10:38:28.608742+0000","last_unstale":"2023-10-19T10:38:29.937558+0000","last_undegraded":"2023-10-19T10:38:29.937558+0000","last_fullsized":"2023-10-19T10:38:29.937558+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:25:06.197277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715001+0000","last_change":"2023-10-19T10:38:27.581569+0000","last_active":"2023-10-19T10:38:30.715001+0000","last_peered":"2023-10-19T10:38:30.715001+0000","last_clean":"2023-10-19T10:38:30.715001+0000","last_became_active":"2023-10-19T10:38:27.581289+0000","last_became_peered":"2023-10-19T10:38:27.581289+0000","last_unstale":"2023-10-19T10:38:30.715001+0000","last_undegraded":"2023-10-19T10:38:30.715001+0000","last_fullsized":"2023-10-19T10:38:30.715001+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:15:51.525358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937519+0000","last_change":"2023-10-19T10:38:28.587724+0000","last_active":"2023-10-19T10:38:29.937519+0000","last_peered":"2023-10-19T10:38:29.937519+0000","last_clean":"2023-10-19T10:38:29.937519+0000","last_became_active":"2023-10-19T10:38:28.587464+0000","last_became_peered":"2023-10-19T10:38:28.587464+0000","last_unstale":"2023-10-19T10:38:29.937519+0000","last_undegraded":"2023-10-19T10:38:29.937519+0000","last_fullsized":"2023-10-19T10:38:29.937519+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:59:51.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148198+0000","last_change":"2023-10-19T10:38:27.587587+0000","last_active":"2023-10-19T10:38:31.148198+0000","last_peered":"2023-10-19T10:38:31.148198+0000","last_clean":"2023-10-19T10:38:31.148198+0000","last_became_active":"2023-10-19T10:38:27.587288+0000","last_became_peered":"2023-10-19T10:38:27.587288+0000","last_unstale":"2023-10-19T10:38:31.148198+0000","last_undegraded":"2023-10-19T10:38:31.148198+0000","last_fullsized":"2023-10-19T10:38:31.148198+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:49:09.429497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592648+0000","last_change":"2023-10-19T10:38:28.585499+0000","last_active":"2023-10-19T10:38:29.592648+0000","last_peered":"2023-10-19T10:38:29.592648+0000","last_clean":"2023-10-19T10:38:29.592648+0000","last_became_active":"2023-10-19T10:38:28.585259+0000","last_became_peered":"2023-10-19T10:38:28.585259+0000","last_unstale":"2023-10-19T10:38:29.592648+0000","last_undegraded":"2023-10-19T10:38:29.592648+0000","last_fullsized":"2023-10-19T10:38:29.592648+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:52:51.318834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937896+0000","last_change":"2023-10-19T10:38:27.584386+0000","last_active":"2023-10-19T10:38:29.937896+0000","last_peered":"2023-10-19T10:38:29.937896+0000","last_clean":"2023-10-19T10:38:29.937896+0000","last_became_active":"2023-10-19T10:38:27.583893+0000","last_became_peered":"2023-10-19T10:38:27.583893+0000","last_unstale":"2023-10-19T10:38:29.937896+0000","last_undegraded":"2023-10-19T10:38:29.937896+0000","last_fullsized":"2023-10-19T10:38:29.937896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:24:18.658643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714999+0000","last_change":"2023-10-19T10:38:28.601147+0000","last_active":"2023-10-19T10:38:30.714999+0000","last_peered":"2023-10-19T10:38:30.714999+0000","last_clean":"2023-10-19T10:38:30.714999+0000","last_became_active":"2023-10-19T10:38:28.600303+0000","last_became_peered":"2023-10-19T10:38:28.600303+0000","last_unstale":"2023-10-19T10:38:30.714999+0000","last_undegraded":"2023-10-19T10:38:30.714999+0000","last_fullsized":"2023-10-19T10:38:30.714999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:00:10.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715125+0000","last_change":"2023-10-19T10:38:27.582927+0000","last_active":"2023-10-19T10:38:30.715125+0000","last_peered":"2023-10-19T10:38:30.715125+0000","last_clean":"2023-10-19T10:38:30.715125+0000","last_became_active":"2023-10-19T10:38:27.582650+0000","last_became_peered":"2023-10-19T10:38:27.582650+0000","last_unstale":"2023-10-19T10:38:30.715125+0000","last_undegraded":"2023-10-19T10:38:30.715125+0000","last_fullsized":"2023-10-19T10:38:30.715125+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:44:23.815810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148514+0000","last_change":"2023-10-19T10:38:28.587263+0000","last_active":"2023-10-19T10:38:31.148514+0000","last_peered":"2023-10-19T10:38:31.148514+0000","last_clean":"2023-10-19T10:38:31.148514+0000","last_became_active":"2023-10-19T10:38:28.586812+0000","last_became_peered":"2023-10-19T10:38:28.586812+0000","last_unstale":"2023-10-19T10:38:31.148514+0000","last_undegraded":"2023-10-19T10:38:31.148514+0000","last_fullsized":"2023-10-19T10:38:31.148514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:35:04.830904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938089+0000","last_change":"2023-10-19T10:38:27.586098+0000","last_active":"2023-10-19T10:38:29.938089+0000","last_peered":"2023-10-19T10:38:29.938089+0000","last_clean":"2023-10-19T10:38:29.938089+0000","last_became_active":"2023-10-19T10:38:27.585600+0000","last_became_peered":"2023-10-19T10:38:27.585600+0000","last_unstale":"2023-10-19T10:38:29.938089+0000","last_undegraded":"2023-10-19T10:38:29.938089+0000","last_fullsized":"2023-10-19T10:38:29.938089+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:35:03.822835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592357+0000","last_change":"2023-10-19T10:38:28.603572+0000","last_active":"2023-10-19T10:38:29.592357+0000","last_peered":"2023-10-19T10:38:29.592357+0000","last_clean":"2023-10-19T10:38:29.592357+0000","last_became_active":"2023-10-19T10:38:28.603273+0000","last_became_peered":"2023-10-19T10:38:28.603273+0000","last_unstale":"2023-10-19T10:38:29.592357+0000","last_undegraded":"2023-10-19T10:38:29.592357+0000","last_fullsized":"2023-10-19T10:38:29.592357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:48:53.289286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148486+0000","last_change":"2023-10-19T10:38:27.589861+0000","last_active":"2023-10-19T10:38:31.148486+0000","last_peered":"2023-10-19T10:38:31.148486+0000","last_clean":"2023-10-19T10:38:31.148486+0000","last_became_active":"2023-10-19T10:38:27.589594+0000","last_became_peered":"2023-10-19T10:38:27.589594+0000","last_unstale":"2023-10-19T10:38:31.148486+0000","last_undegraded":"2023-10-19T10:38:31.148486+0000","last_fullsized":"2023-10-19T10:38:31.148486+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:32:44.914326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148182+0000","last_change":"2023-10-19T10:38:28.593415+0000","last_active":"2023-10-19T10:38:31.148182+0000","last_peered":"2023-10-19T10:38:31.148182+0000","last_clean":"2023-10-19T10:38:31.148182+0000","last_became_active":"2023-10-19T10:38:28.593072+0000","last_became_peered":"2023-10-19T10:38:28.593072+0000","last_unstale":"2023-10-19T10:38:31.148182+0000","last_undegraded":"2023-10-19T10:38:31.148182+0000","last_fullsized":"2023-10-19T10:38:31.148182+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:57.487371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589438+0000","last_change":"2023-10-19T10:38:27.587308+0000","last_active":"2023-10-19T10:38:29.589438+0000","last_peered":"2023-10-19T10:38:29.589438+0000","last_clean":"2023-10-19T10:38:29.589438+0000","last_became_active":"2023-10-19T10:38:27.586928+0000","last_became_peered":"2023-10-19T10:38:27.586928+0000","last_unstale":"2023-10-19T10:38:29.589438+0000","last_undegraded":"2023-10-19T10:38:29.589438+0000","last_fullsized":"2023-10-19T10:38:29.589438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:20:34.220998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593428+0000","last_change":"2023-10-19T10:38:28.588339+0000","last_active":"2023-10-19T10:38:29.593428+0000","last_peered":"2023-10-19T10:38:29.593428+0000","last_clean":"2023-10-19T10:38:29.593428+0000","last_became_active":"2023-10-19T10:38:28.588090+0000","last_became_peered":"2023-10-19T10:38:28.588090+0000","last_unstale":"2023-10-19T10:38:29.593428+0000","last_undegraded":"2023-10-19T10:38:29.593428+0000","last_fullsized":"2023-10-19T10:38:29.593428+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:05:54.866959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592484+0000","last_change":"2023-10-19T10:38:27.588963+0000","last_active":"2023-10-19T10:38:29.592484+0000","last_peered":"2023-10-19T10:38:29.592484+0000","last_clean":"2023-10-19T10:38:29.592484+0000","last_became_active":"2023-10-19T10:38:27.588617+0000","last_became_peered":"2023-10-19T10:38:27.588617+0000","last_unstale":"2023-10-19T10:38:29.592484+0000","last_undegraded":"2023-10-19T10:38:29.592484+0000","last_fullsized":"2023-10-19T10:38:29.592484+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:24:18.658643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937651+0000","last_change":"2023-10-19T10:38:28.590252+0000","last_active":"2023-10-19T10:38:29.937651+0000","last_peered":"2023-10-19T10:38:29.937651+0000","last_clean":"2023-10-19T10:38:29.937651+0000","last_became_active":"2023-10-19T10:38:28.589980+0000","last_became_peered":"2023-10-19T10:38:28.589980+0000","last_unstale":"2023-10-19T10:38:29.937651+0000","last_undegraded":"2023-10-19T10:38:29.937651+0000","last_fullsized":"2023-10-19T10:38:29.937651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:53:14.764775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148581+0000","last_change":"2023-10-19T10:38:27.584475+0000","last_active":"2023-10-19T10:38:31.148581+0000","last_peered":"2023-10-19T10:38:31.148581+0000","last_clean":"2023-10-19T10:38:31.148581+0000","last_became_active":"2023-10-19T10:38:27.584175+0000","last_became_peered":"2023-10-19T10:38:27.584175+0000","last_unstale":"2023-10-19T10:38:31.148581+0000","last_undegraded":"2023-10-19T10:38:31.148581+0000","last_fullsized":"2023-10-19T10:38:31.148581+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:49:09.429497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592024+0000","last_change":"2023-10-19T10:38:28.594638+0000","last_active":"2023-10-19T10:38:29.592024+0000","last_peered":"2023-10-19T10:38:29.592024+0000","last_clean":"2023-10-19T10:38:29.592024+0000","last_became_active":"2023-10-19T10:38:28.594308+0000","last_became_peered":"2023-10-19T10:38:28.594308+0000","last_unstale":"2023-10-19T10:38:29.592024+0000","last_undegraded":"2023-10-19T10:38:29.592024+0000","last_fullsized":"2023-10-19T10:38:29.592024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:48:54.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592173+0000","last_change":"2023-10-19T10:38:27.589618+0000","last_active":"2023-10-19T10:38:29.592173+0000","last_peered":"2023-10-19T10:38:29.592173+0000","last_clean":"2023-10-19T10:38:29.592173+0000","last_became_active":"2023-10-19T10:38:27.589322+0000","last_became_peered":"2023-10-19T10:38:27.589322+0000","last_unstale":"2023-10-19T10:38:29.592173+0000","last_undegraded":"2023-10-19T10:38:29.592173+0000","last_fullsized":"2023-10-19T10:38:29.592173+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:13:34.146672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148824+0000","last_change":"2023-10-19T10:38:28.581775+0000","last_active":"2023-10-19T10:38:31.148824+0000","last_peered":"2023-10-19T10:38:31.148824+0000","last_clean":"2023-10-19T10:38:31.148824+0000","last_became_active":"2023-10-19T10:38:28.581509+0000","last_became_peered":"2023-10-19T10:38:28.581509+0000","last_unstale":"2023-10-19T10:38:31.148824+0000","last_undegraded":"2023-10-19T10:38:31.148824+0000","last_fullsized":"2023-10-19T10:38:31.148824+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:48:09.095111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937828+0000","last_change":"2023-10-19T10:38:27.584538+0000","last_active":"2023-10-19T10:38:29.937828+0000","last_peered":"2023-10-19T10:38:29.937828+0000","last_clean":"2023-10-19T10:38:29.937828+0000","last_became_active":"2023-10-19T10:38:27.584243+0000","last_became_peered":"2023-10-19T10:38:27.584243+0000","last_unstale":"2023-10-19T10:38:29.937828+0000","last_undegraded":"2023-10-19T10:38:29.937828+0000","last_fullsized":"2023-10-19T10:38:29.937828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:00:34.280277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589538+0000","last_change":"2023-10-19T10:38:28.589270+0000","last_active":"2023-10-19T10:38:29.589538+0000","last_peered":"2023-10-19T10:38:29.589538+0000","last_clean":"2023-10-19T10:38:29.589538+0000","last_became_active":"2023-10-19T10:38:28.588950+0000","last_became_peered":"2023-10-19T10:38:28.588950+0000","last_unstale":"2023-10-19T10:38:29.589538+0000","last_undegraded":"2023-10-19T10:38:29.589538+0000","last_fullsized":"2023-10-19T10:38:29.589538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:41:29.958037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589459+0000","last_change":"2023-10-19T10:38:27.591280+0000","last_active":"2023-10-19T10:38:29.589459+0000","last_peered":"2023-10-19T10:38:29.589459+0000","last_clean":"2023-10-19T10:38:29.589459+0000","last_became_active":"2023-10-19T10:38:27.590962+0000","last_became_peered":"2023-10-19T10:38:27.590962+0000","last_unstale":"2023-10-19T10:38:29.589459+0000","last_undegraded":"2023-10-19T10:38:29.589459+0000","last_fullsized":"2023-10-19T10:38:29.589459+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:00:03.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938056+0000","last_change":"2023-10-19T10:38:28.596649+0000","last_active":"2023-10-19T10:38:29.938056+0000","last_peered":"2023-10-19T10:38:29.938056+0000","last_clean":"2023-10-19T10:38:29.938056+0000","last_became_active":"2023-10-19T10:38:28.596384+0000","last_became_peered":"2023-10-19T10:38:28.596384+0000","last_unstale":"2023-10-19T10:38:29.938056+0000","last_undegraded":"2023-10-19T10:38:29.938056+0000","last_fullsized":"2023-10-19T10:38:29.938056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:32:45.922396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715110+0000","last_change":"2023-10-19T10:38:27.581103+0000","last_active":"2023-10-19T10:38:30.715110+0000","last_peered":"2023-10-19T10:38:30.715110+0000","last_clean":"2023-10-19T10:38:30.715110+0000","last_became_active":"2023-10-19T10:38:27.579968+0000","last_became_peered":"2023-10-19T10:38:27.579968+0000","last_unstale":"2023-10-19T10:38:30.715110+0000","last_undegraded":"2023-10-19T10:38:30.715110+0000","last_fullsized":"2023-10-19T10:38:30.715110+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:50:58.315710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592975+0000","last_change":"2023-10-19T10:38:28.589843+0000","last_active":"2023-10-19T10:38:29.592975+0000","last_peered":"2023-10-19T10:38:29.592975+0000","last_clean":"2023-10-19T10:38:29.592975+0000","last_became_active":"2023-10-19T10:38:28.589496+0000","last_became_peered":"2023-10-19T10:38:28.589496+0000","last_unstale":"2023-10-19T10:38:29.592975+0000","last_undegraded":"2023-10-19T10:38:29.592975+0000","last_fullsized":"2023-10-19T10:38:29.592975+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:46:28.716699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148962+0000","last_change":"2023-10-19T10:38:27.588169+0000","last_active":"2023-10-19T10:38:31.148962+0000","last_peered":"2023-10-19T10:38:31.148962+0000","last_clean":"2023-10-19T10:38:31.148962+0000","last_became_active":"2023-10-19T10:38:27.587837+0000","last_became_peered":"2023-10-19T10:38:27.587837+0000","last_unstale":"2023-10-19T10:38:31.148962+0000","last_undegraded":"2023-10-19T10:38:31.148962+0000","last_fullsized":"2023-10-19T10:38:31.148962+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:05:32.450768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593126+0000","last_change":"2023-10-19T10:38:28.593434+0000","last_active":"2023-10-19T10:38:29.593126+0000","last_peered":"2023-10-19T10:38:29.593126+0000","last_clean":"2023-10-19T10:38:29.593126+0000","last_became_active":"2023-10-19T10:38:28.593112+0000","last_became_peered":"2023-10-19T10:38:28.593112+0000","last_unstale":"2023-10-19T10:38:29.593126+0000","last_undegraded":"2023-10-19T10:38:29.593126+0000","last_fullsized":"2023-10-19T10:38:29.593126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:53:27.890189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592402+0000","last_change":"2023-10-19T10:38:27.585994+0000","last_active":"2023-10-19T10:38:29.592402+0000","last_peered":"2023-10-19T10:38:29.592402+0000","last_clean":"2023-10-19T10:38:29.592402+0000","last_became_active":"2023-10-19T10:38:27.585697+0000","last_became_peered":"2023-10-19T10:38:27.585697+0000","last_unstale":"2023-10-19T10:38:29.592402+0000","last_undegraded":"2023-10-19T10:38:29.592402+0000","last_fullsized":"2023-10-19T10:38:29.592402+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:08.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592155+0000","last_change":"2023-10-19T10:38:28.603272+0000","last_active":"2023-10-19T10:38:29.592155+0000","last_peered":"2023-10-19T10:38:29.592155+0000","last_clean":"2023-10-19T10:38:29.592155+0000","last_became_active":"2023-10-19T10:38:28.602920+0000","last_became_peered":"2023-10-19T10:38:28.602920+0000","last_unstale":"2023-10-19T10:38:29.592155+0000","last_undegraded":"2023-10-19T10:38:29.592155+0000","last_fullsized":"2023-10-19T10:38:29.592155+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:59:51.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592554+0000","last_change":"2023-10-19T10:38:27.590415+0000","last_active":"2023-10-19T10:38:29.592554+0000","last_peered":"2023-10-19T10:38:29.592554+0000","last_clean":"2023-10-19T10:38:29.592554+0000","last_became_active":"2023-10-19T10:38:27.589481+0000","last_became_peered":"2023-10-19T10:38:27.589481+0000","last_unstale":"2023-10-19T10:38:29.592554+0000","last_undegraded":"2023-10-19T10:38:29.592554+0000","last_fullsized":"2023-10-19T10:38:29.592554+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:56.479302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605441+0000","last_change":"2023-10-19T10:38:28.584626+0000","last_active":"2023-10-19T10:38:30.605441+0000","last_peered":"2023-10-19T10:38:30.605441+0000","last_clean":"2023-10-19T10:38:30.605441+0000","last_became_active":"2023-10-19T10:38:28.584346+0000","last_became_peered":"2023-10-19T10:38:28.584346+0000","last_unstale":"2023-10-19T10:38:30.605441+0000","last_undegraded":"2023-10-19T10:38:30.605441+0000","last_fullsized":"2023-10-19T10:38:30.605441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:48:30.382781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148088+0000","last_change":"2023-10-19T10:38:27.590514+0000","last_active":"2023-10-19T10:38:31.148088+0000","last_peered":"2023-10-19T10:38:31.148088+0000","last_clean":"2023-10-19T10:38:31.148088+0000","last_became_active":"2023-10-19T10:38:27.590194+0000","last_became_peered":"2023-10-19T10:38:27.590194+0000","last_unstale":"2023-10-19T10:38:31.148088+0000","last_undegraded":"2023-10-19T10:38:31.148088+0000","last_fullsized":"2023-10-19T10:38:31.148088+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:39:06.607750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589185+0000","last_change":"2023-10-19T10:38:28.590033+0000","last_active":"2023-10-19T10:38:29.589185+0000","last_peered":"2023-10-19T10:38:29.589185+0000","last_clean":"2023-10-19T10:38:29.589185+0000","last_became_active":"2023-10-19T10:38:28.589739+0000","last_became_peered":"2023-10-19T10:38:28.589739+0000","last_unstale":"2023-10-19T10:38:29.589185+0000","last_undegraded":"2023-10-19T10:38:29.589185+0000","last_fullsized":"2023-10-19T10:38:29.589185+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:50:27.417488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937527+0000","last_change":"2023-10-19T10:38:27.586164+0000","last_active":"2023-10-19T10:38:29.937527+0000","last_peered":"2023-10-19T10:38:29.937527+0000","last_clean":"2023-10-19T10:38:29.937527+0000","last_became_active":"2023-10-19T10:38:27.585838+0000","last_became_peered":"2023-10-19T10:38:27.585838+0000","last_unstale":"2023-10-19T10:38:29.937527+0000","last_undegraded":"2023-10-19T10:38:29.937527+0000","last_fullsized":"2023-10-19T10:38:29.937527+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:13:34.146672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937494+0000","last_change":"2023-10-19T10:38:28.597364+0000","last_active":"2023-10-19T10:38:29.937494+0000","last_peered":"2023-10-19T10:38:29.937494+0000","last_clean":"2023-10-19T10:38:29.937494+0000","last_became_active":"2023-10-19T10:38:28.597091+0000","last_became_peered":"2023-10-19T10:38:28.597091+0000","last_unstale":"2023-10-19T10:38:29.937494+0000","last_undegraded":"2023-10-19T10:38:29.937494+0000","last_fullsized":"2023-10-19T10:38:29.937494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:14:35.055092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148896+0000","last_change":"2023-10-19T10:38:27.584351+0000","last_active":"2023-10-19T10:38:31.148896+0000","last_peered":"2023-10-19T10:38:31.148896+0000","last_clean":"2023-10-19T10:38:31.148896+0000","last_became_active":"2023-10-19T10:38:27.583963+0000","last_became_peered":"2023-10-19T10:38:27.583963+0000","last_unstale":"2023-10-19T10:38:31.148896+0000","last_undegraded":"2023-10-19T10:38:31.148896+0000","last_fullsized":"2023-10-19T10:38:31.148896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:44:41.861501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592347+0000","last_change":"2023-10-19T10:38:28.593399+0000","last_active":"2023-10-19T10:38:29.592347+0000","last_peered":"2023-10-19T10:38:29.592347+0000","last_clean":"2023-10-19T10:38:29.592347+0000","last_became_active":"2023-10-19T10:38:28.593085+0000","last_became_peered":"2023-10-19T10:38:28.593085+0000","last_unstale":"2023-10-19T10:38:29.592347+0000","last_undegraded":"2023-10-19T10:38:29.592347+0000","last_fullsized":"2023-10-19T10:38:29.592347+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:46:40.058837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148672+0000","last_change":"2023-10-19T10:38:27.590766+0000","last_active":"2023-10-19T10:38:31.148672+0000","last_peered":"2023-10-19T10:38:31.148672+0000","last_clean":"2023-10-19T10:38:31.148672+0000","last_became_active":"2023-10-19T10:38:27.590488+0000","last_became_peered":"2023-10-19T10:38:27.590488+0000","last_unstale":"2023-10-19T10:38:31.148672+0000","last_undegraded":"2023-10-19T10:38:31.148672+0000","last_fullsized":"2023-10-19T10:38:31.148672+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:15:52.161127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593455+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:38:29.593455+0000","last_peered":"2023-10-19T10:38:29.593455+0000","last_clean":"2023-10-19T10:38:29.593455+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:38:29.593455+0000","last_undegraded":"2023-10-19T10:38:29.593455+0000","last_fullsized":"2023-10-19T10:38:29.593455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:07:51.595740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.592306+0000","last_change":"2023-10-19T10:38:28.603225+0000","last_active":"2023-10-19T10:38:29.592306+0000","last_peered":"2023-10-19T10:38:29.592306+0000","last_clean":"2023-10-19T10:38:29.592306+0000","last_became_active":"2023-10-19T10:38:28.602887+0000","last_became_peered":"2023-10-19T10:38:28.602887+0000","last_unstale":"2023-10-19T10:38:29.592306+0000","last_undegraded":"2023-10-19T10:38:29.592306+0000","last_fullsized":"2023-10-19T10:38:29.592306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:53:14.764775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.715005+0000","last_change":"2023-10-19T10:38:27.591306+0000","last_active":"2023-10-19T10:38:30.715005+0000","last_peered":"2023-10-19T10:38:30.715005+0000","last_clean":"2023-10-19T10:38:30.715005+0000","last_became_active":"2023-10-19T10:38:27.591039+0000","last_became_peered":"2023-10-19T10:38:27.591039+0000","last_unstale":"2023-10-19T10:38:30.715005+0000","last_undegraded":"2023-10-19T10:38:30.715005+0000","last_fullsized":"2023-10-19T10:38:30.715005+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:13:32.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605657+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:38:30.605657+0000","last_peered":"2023-10-19T10:38:30.605657+0000","last_clean":"2023-10-19T10:38:30.605657+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:38:30.605657+0000","last_undegraded":"2023-10-19T10:38:30.605657+0000","last_fullsized":"2023-10-19T10:38:30.605657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:43:55.690108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148046+0000","last_change":"2023-10-19T10:38:28.593159+0000","last_active":"2023-10-19T10:38:31.148046+0000","last_peered":"2023-10-19T10:38:31.148046+0000","last_clean":"2023-10-19T10:38:31.148046+0000","last_became_active":"2023-10-19T10:38:28.592848+0000","last_became_peered":"2023-10-19T10:38:28.592848+0000","last_unstale":"2023-10-19T10:38:31.148046+0000","last_undegraded":"2023-10-19T10:38:31.148046+0000","last_fullsized":"2023-10-19T10:38:31.148046+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:08:14.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605389+0000","last_change":"2023-10-19T10:38:27.586732+0000","last_active":"2023-10-19T10:38:30.605389+0000","last_peered":"2023-10-19T10:38:30.605389+0000","last_clean":"2023-10-19T10:38:30.605389+0000","last_became_active":"2023-10-19T10:38:27.586225+0000","last_became_peered":"2023-10-19T10:38:27.586225+0000","last_unstale":"2023-10-19T10:38:30.605389+0000","last_undegraded":"2023-10-19T10:38:30.605389+0000","last_fullsized":"2023-10-19T10:38:30.605389+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:45:14.987989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148970+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:38:31.148970+0000","last_peered":"2023-10-19T10:38:31.148970+0000","last_clean":"2023-10-19T10:38:31.148970+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:38:31.148970+0000","last_undegraded":"2023-10-19T10:38:31.148970+0000","last_fullsized":"2023-10-19T10:38:31.148970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:45:22.810611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714772+0000","last_change":"2023-10-19T10:38:28.598900+0000","last_active":"2023-10-19T10:38:30.714772+0000","last_peered":"2023-10-19T10:38:30.714772+0000","last_clean":"2023-10-19T10:38:30.714772+0000","last_became_active":"2023-10-19T10:38:28.598635+0000","last_became_peered":"2023-10-19T10:38:28.598635+0000","last_unstale":"2023-10-19T10:38:30.714772+0000","last_undegraded":"2023-10-19T10:38:30.714772+0000","last_fullsized":"2023-10-19T10:38:30.714772+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T10:41:18.698468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937958+0000","last_change":"2023-10-19T10:38:27.593111+0000","last_active":"2023-10-19T10:38:29.937958+0000","last_peered":"2023-10-19T10:38:29.937958+0000","last_clean":"2023-10-19T10:38:29.937958+0000","last_became_active":"2023-10-19T10:38:27.592822+0000","last_became_peered":"2023-10-19T10:38:27.592822+0000","last_unstale":"2023-10-19T10:38:29.937958+0000","last_undegraded":"2023-10-19T10:38:29.937958+0000","last_fullsized":"2023-10-19T10:38:29.937958+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:12:20.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605407+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:38:30.605407+0000","last_peered":"2023-10-19T10:38:30.605407+0000","last_clean":"2023-10-19T10:38:30.605407+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:38:30.605407+0000","last_undegraded":"2023-10-19T10:38:30.605407+0000","last_fullsized":"2023-10-19T10:38:30.605407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:30:49.335478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.593466+0000","last_change":"2023-10-19T10:38:28.587724+0000","last_active":"2023-10-19T10:38:29.593466+0000","last_peered":"2023-10-19T10:38:29.593466+0000","last_clean":"2023-10-19T10:38:29.593466+0000","last_became_active":"2023-10-19T10:38:28.587039+0000","last_became_peered":"2023-10-19T10:38:28.587039+0000","last_unstale":"2023-10-19T10:38:29.593466+0000","last_undegraded":"2023-10-19T10:38:29.593466+0000","last_fullsized":"2023-10-19T10:38:29.593466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:44:13.072808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.714344+0000","last_change":"2023-10-19T10:38:27.588946+0000","last_active":"2023-10-19T10:38:30.714344+0000","last_peered":"2023-10-19T10:38:30.714344+0000","last_clean":"2023-10-19T10:38:30.714344+0000","last_became_active":"2023-10-19T10:38:27.588664+0000","last_became_peered":"2023-10-19T10:38:27.588664+0000","last_unstale":"2023-10-19T10:38:30.714344+0000","last_undegraded":"2023-10-19T10:38:30.714344+0000","last_fullsized":"2023-10-19T10:38:30.714344+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:00:34.280277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148071+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:38:31.148071+0000","last_peered":"2023-10-19T10:38:31.148071+0000","last_clean":"2023-10-19T10:38:31.148071+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:38:31.148071+0000","last_undegraded":"2023-10-19T10:38:31.148071+0000","last_fullsized":"2023-10-19T10:38:31.148071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:31:07.514042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.937705+0000","last_change":"2023-10-19T10:38:28.589427+0000","last_active":"2023-10-19T10:38:29.937705+0000","last_peered":"2023-10-19T10:38:29.937705+0000","last_clean":"2023-10-19T10:38:29.937705+0000","last_became_active":"2023-10-19T10:38:28.588927+0000","last_became_peered":"2023-10-19T10:38:28.588927+0000","last_unstale":"2023-10-19T10:38:29.937705+0000","last_undegraded":"2023-10-19T10:38:29.937705+0000","last_fullsized":"2023-10-19T10:38:29.937705+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:09.216870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148623+0000","last_change":"2023-10-19T10:38:27.590779+0000","last_active":"2023-10-19T10:38:31.148623+0000","last_peered":"2023-10-19T10:38:31.148623+0000","last_clean":"2023-10-19T10:38:31.148623+0000","last_became_active":"2023-10-19T10:38:27.590482+0000","last_became_peered":"2023-10-19T10:38:27.590482+0000","last_unstale":"2023-10-19T10:38:31.148623+0000","last_undegraded":"2023-10-19T10:38:31.148623+0000","last_fullsized":"2023-10-19T10:38:31.148623+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:26:45.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.589099+0000","last_change":"2023-10-19T10:36:50.164996+0000","last_active":"2023-10-19T10:38:29.589099+0000","last_peered":"2023-10-19T10:38:29.589099+0000","last_clean":"2023-10-19T10:38:29.589099+0000","last_became_active":"2023-10-19T10:36:50.164628+0000","last_became_peered":"2023-10-19T10:36:50.164628+0000","last_unstale":"2023-10-19T10:38:29.589099+0000","last_undegraded":"2023-10-19T10:38:29.589099+0000","last_fullsized":"2023-10-19T10:38:29.589099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:13:06.119282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148343+0000","last_change":"2023-10-19T10:38:28.582430+0000","last_active":"2023-10-19T10:38:31.148343+0000","last_peered":"2023-10-19T10:38:31.148343+0000","last_clean":"2023-10-19T10:38:31.148343+0000","last_became_active":"2023-10-19T10:38:28.582174+0000","last_became_peered":"2023-10-19T10:38:28.582174+0000","last_unstale":"2023-10-19T10:38:31.148343+0000","last_undegraded":"2023-10-19T10:38:31.148343+0000","last_fullsized":"2023-10-19T10:38:31.148343+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:39:07.615819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:30.605422+0000","last_change":"2023-10-19T10:38:27.584088+0000","last_active":"2023-10-19T10:38:30.605422+0000","last_peered":"2023-10-19T10:38:30.605422+0000","last_clean":"2023-10-19T10:38:30.605422+0000","last_became_active":"2023-10-19T10:38:27.583803+0000","last_became_peered":"2023-10-19T10:38:27.583803+0000","last_unstale":"2023-10-19T10:38:30.605422+0000","last_undegraded":"2023-10-19T10:38:30.605422+0000","last_fullsized":"2023-10-19T10:38:30.605422+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:05:16.906120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148762+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:38:31.148762+0000","last_peered":"2023-10-19T10:38:31.148762+0000","last_clean":"2023-10-19T10:38:31.148762+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:38:31.148762+0000","last_undegraded":"2023-10-19T10:38:31.148762+0000","last_fullsized":"2023-10-19T10:38:31.148762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:03:55.050485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:29.938027+0000","last_change":"2023-10-19T10:38:28.589570+0000","last_active":"2023-10-19T10:38:29.938027+0000","last_peered":"2023-10-19T10:38:29.938027+0000","last_clean":"2023-10-19T10:38:29.938027+0000","last_became_active":"2023-10-19T10:38:28.589288+0000","last_became_peered":"2023-10-19T10:38:28.589288+0000","last_unstale":"2023-10-19T10:38:29.938027+0000","last_undegraded":"2023-10-19T10:38:29.938027+0000","last_fullsized":"2023-10-19T10:38:29.938027+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:27.561034+0000","last_clean_scrub_stamp":"2023-10-19T10:38:27.561034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:39:07.615819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.149113+0000","last_change":"2023-10-19T10:38:27.591386+0000","last_active":"2023-10-19T10:38:31.149113+0000","last_peered":"2023-10-19T10:38:31.149113+0000","last_clean":"2023-10-19T10:38:31.149113+0000","last_became_active":"2023-10-19T10:38:27.591057+0000","last_became_peered":"2023-10-19T10:38:27.591057+0000","last_unstale":"2023-10-19T10:38:31.149113+0000","last_undegraded":"2023-10-19T10:38:31.149113+0000","last_fullsized":"2023-10-19T10:38:31.149113+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:38:26.552965+0000","last_clean_scrub_stamp":"2023-10-19T10:38:26.552965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:47:00.210895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148284+0000","last_change":"2023-10-19T10:36:50.167910+0000","last_active":"2023-10-19T10:38:31.148284+0000","last_peered":"2023-10-19T10:38:31.148284+0000","last_clean":"2023-10-19T10:38:31.148284+0000","last_became_active":"2023-10-19T10:36:50.167571+0000","last_became_peered":"2023-10-19T10:36:50.167571+0000","last_unstale":"2023-10-19T10:38:31.148284+0000","last_undegraded":"2023-10-19T10:38:31.148284+0000","last_fullsized":"2023-10-19T10:38:31.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T22:25:07.696839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":211,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-19T10:38:31.148995+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:38:31.148995+0000","last_peered":"2023-10-19T10:38:31.148995+0000","last_clean":"2023-10-19T10:38:31.148995+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:38:31.148995+0000","last_undegraded":"2023-10-19T10:38:31.148995+0000","last_fullsized":"2023-10-19T10:38:31.148995+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:14:49.238856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":808954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.436},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.316}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.498}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":495616,"data_stored":219168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.627},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":211593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.376}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.626},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.495},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.42799999999999999}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":213135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.627},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":499712,"data_stored":219134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63}]},{"osd":7,"last update":"Thu Oct 19 10:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":503808,"data_stored":220130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.308},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.628}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":491520,"data_stored":218566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.624},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.38}]},{"osd":3,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.752}]},{"osd":7,"last update":"Thu Oct 19 10:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1085440,"data_stored":808898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.377},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.63},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:38:36.393 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-19T10:38:36.394 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-19T10:38:36.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 Processor -- start 2023-10-19T10:38:36.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 -- start start 2023-10-19T10:38:36.544 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fba480b4070 0x7fba480b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba4815cdf0 0x7fba480cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba480d3480 con 0x7fba4815c960 2023-10-19T10:38:36.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba480b4990 con 0x7fba480b4070 2023-10-19T10:38:36.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.538+0000 7fba4cecb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4815d280 con 0x7fba4815cdf0 2023-10-19T10:38:36.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba3dd9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fba4815c960 0x7fba480d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58924/0 (socket says 172.21.15.50:58924) 2023-10-19T10:38:36.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba3dd9b640 1 -- 172.21.15.50:0/3900993740 learned_addr learned my addr 172.21.15.50:0/3900993740 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:36.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba46d9d640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba4815cdf0 0x7fba480cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368985226 0 0) 0x7fba480d3480 con 0x7fba4815c960 2023-10-19T10:38:36.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba34003520 con 0x7fba4815c960 2023-10-19T10:38:36.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba4659c640 1 --2- 172.21.15.50:0/3900993740 >> v2:172.21.15.183:3300/0 conn(0x7fba480b4070 0x7fba480b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1831666755 0 0) 0x7fba34003520 con 0x7fba4815c960 2023-10-19T10:38:36.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba4815cdf0 msgr2=0x7fba480cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:36.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba4815cdf0 0x7fba480cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 >> v2:172.21.15.183:3300/0 conn(0x7fba480b4070 msgr2=0x7fba480b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:36.547 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 --2- 172.21.15.50:0/3900993740 >> v2:172.21.15.183:3300/0 conn(0x7fba480b4070 0x7fba480b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4815d560 con 0x7fba4815c960 2023-10-19T10:38:36.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fba30001690 con 0x7fba4815c960 2023-10-19T10:38:36.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 >> v1:172.21.15.50:6789/0 conn(0x7fba4815c960 legacy=0x7fba480d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba34003970 0x7fba34003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba340042d0 0x7fba340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba34004e10 0x7fba34005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480b4990 con 0x7fba340042d0 2023-10-19T10:38:36.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4815d280 con 0x7fba34004e10 2023-10-19T10:38:36.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.539+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3400bd00 con 0x7fba34003970 2023-10-19T10:38:36.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba340042d0 0x7fba340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba4659c640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba34003970 0x7fba34003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba46d9d640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba34004e10 0x7fba34005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 -- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba34003970 msgr2=0x7fba34003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:36.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba34003970 0x7fba34003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 -- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba34004e10 msgr2=0x7fba34005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:36.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba34004e10 0x7fba34005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 -- 172.21.15.50:0/3900993740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba34003520 con 0x7fba340042d0 2023-10-19T10:38:36.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.540+0000 7fba3dd9b640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba340042d0 0x7fba340047c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fba480b1f00 tx=0x7fba340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3366f08e88db2cb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30005ad0 con 0x7fba340042d0 2023-10-19T10:38:36.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30005c50 con 0x7fba340042d0 2023-10-19T10:38:36.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba45d9b640 1 -- 172.21.15.50:0/3900993740 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3002c8a0 con 0x7fba340042d0 2023-10-19T10:38:36.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba4cecb640 1 -- 172.21.15.50:0/3900993740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba340042d0 msgr2=0x7fba340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3900993740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba340042d0 0x7fba340047c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fba480b1f00 tx=0x7fba340057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba46d9d640 1 -- 172.21.15.50:0/3900993740 reap_dead start 2023-10-19T10:38:36.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba4cecb640 1 -- 172.21.15.50:0/3900993740 shutdown_connections 2023-10-19T10:38:36.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba4cecb640 1 -- 172.21.15.50:0/3900993740 >> 172.21.15.50:0/3900993740 conn(0x7fba4809b970 msgr2=0x7fba480d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:36.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba4cecb640 1 -- 172.21.15.50:0/3900993740 shutdown_connections 2023-10-19T10:38:36.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.541+0000 7fba4cecb640 1 -- 172.21.15.50:0/3900993740 wait complete. 2023-10-19T10:38:36.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.543+0000 7fba4cecb640 1 Processor -- start 2023-10-19T10:38:36.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.543+0000 7fba4cecb640 1 -- start start 2023-10-19T10:38:36.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.543+0000 7fba4cecb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba340042d0 0x7fba48158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.543+0000 7fba4cecb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba4815eec0 0x7fba481588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.543+0000 7fba4cecb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 0x7fba480c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.543+0000 7fba4cecb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480c8f00 con 0x7fba480c8530 2023-10-19T10:38:36.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba4cecb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480cae20 con 0x7fba4815eec0 2023-10-19T10:38:36.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba4cecb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba48158df0 con 0x7fba340042d0 2023-10-19T10:38:36.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 0x7fba480c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 0x7fba480c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42014/0 (socket says 172.21.15.50:42014) 2023-10-19T10:38:36.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 -- 172.21.15.50:0/3707080365 learned_addr learned my addr 172.21.15.50:0/3707080365 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:36.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba4659c640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba340042d0 0x7fba48158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba3dd9b640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba4815eec0 0x7fba481588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 -- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba340042d0 msgr2=0x7fba48158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba340042d0 0x7fba48158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 -- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba4815eec0 msgr2=0x7fba481588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba4815eec0 0x7fba481588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 -- 172.21.15.50:0/3707080365 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4815f3a0 con 0x7fba480c8530 2023-10-19T10:38:36.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba4659c640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba340042d0 0x7fba48158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:36.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba3dd9b640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba4815eec0 0x7fba481588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:36.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba46d9d640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 0x7fba480c89c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fba480d05a0 tx=0x7fba4000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000e040 con 0x7fba480c8530 2023-10-19T10:38:36.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4000f040 con 0x7fba480c8530 2023-10-19T10:38:36.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4815f680 con 0x7fba480c8530 2023-10-19T10:38:36.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40015430 con 0x7fba480c8530 2023-10-19T10:38:36.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.544+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4815fc40 con 0x7fba480c8530 2023-10-19T10:38:36.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.545+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba480d09b0 con 0x7fba480c8530 2023-10-19T10:38:36.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.546+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fba40015590 con 0x7fba480c8530 2023-10-19T10:38:36.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.546+0000 7fba3f7fe640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba2806a080 0x7fba2806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.546+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fba40088d30 con 0x7fba480c8530 2023-10-19T10:38:36.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.546+0000 7fba4659c640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba2806a080 0x7fba2806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.547+0000 7fba4659c640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba2806a080 0x7fba2806c540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fba480c82f0 tx=0x7fba38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.551+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fba400542a0 con 0x7fba480c8530 2023-10-19T10:38:36.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.794+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fba4815fdd0 con 0x7fba480c8530 2023-10-19T10:38:36.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.797+0000 7fba3f7fe640 1 -- 172.21.15.50:0/3707080365 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fba400578c0 con 0x7fba480c8530 2023-10-19T10:38:36.800 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2023-10-19T10:38:36.801 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCOBjFlR3SvKhAAQGRIzy2kDJzeLJAJbvCD9Q== 2023-10-19T10:38:36.801 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2023-10-19T10:38:36.801 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2023-10-19T10:38:36.801 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-19T10:38:36.803 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2023-10-19T10:38:36.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba2806a080 msgr2=0x7fba2806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba2806a080 0x7fba2806c540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fba480c82f0 tx=0x7fba38008040 comp rx=0 tx=0).stop 2023-10-19T10:38:36.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 msgr2=0x7fba480c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 0x7fba480c89c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fba480d05a0 tx=0x7fba4000f580 comp rx=0 tx=0).stop 2023-10-19T10:38:36.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 shutdown_connections 2023-10-19T10:38:36.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fba2806a080 0x7fba2806c540 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba480c8530 0x7fba480c89c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba4815eec0 0x7fba481588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 --2- 172.21.15.50:0/3707080365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba340042d0 0x7fba48158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 >> 172.21.15.50:0/3707080365 conn(0x7fba4809b970 msgr2=0x7fba4806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:36.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 shutdown_connections 2023-10-19T10:38:36.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.799+0000 7fba4cecb640 1 -- 172.21.15.50:0/3707080365 wait complete. 2023-10-19T10:38:36.812 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:36.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.915+0000 7f3267677640 1 Processor -- start 2023-10-19T10:38:36.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.915+0000 7f3267677640 1 -- start start 2023-10-19T10:38:36.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3267677640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f326015c570 0x7f32600d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3267677640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3267677640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f326006b760 0x7f32600d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3267677640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600cde60 con 0x7f326015c570 2023-10-19T10:38:36.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3267677640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326006aa30 con 0x7f326006b760 2023-10-19T10:38:36.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3267677640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326006b2d0 con 0x7f326015c950 2023-10-19T10:38:36.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3265413640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f326015c570 0x7f32600d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3265413640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f326015c570 0x7f32600d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38330/0 (socket says 172.21.15.50:38330) 2023-10-19T10:38:36.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.916+0000 7f3265413640 1 -- 172.21.15.50:0/2209444967 learned_addr learned my addr 172.21.15.50:0/2209444967 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:36.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265c14640 1 --2- 172.21.15.50:0/2209444967 >> v2:172.21.15.183:3300/0 conn(0x7f326006b760 0x7f32600d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3264c12640 1 --2- 172.21.15.50:0/2209444967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265413640 1 -- 172.21.15.50:0/2209444967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 msgr2=0x7f32600cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:36.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265413640 1 --2- 172.21.15.50:0/2209444967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265413640 1 -- 172.21.15.50:0/2209444967 >> v2:172.21.15.183:3300/0 conn(0x7f326006b760 msgr2=0x7f32600d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:36.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265413640 1 --2- 172.21.15.50:0/2209444967 >> v2:172.21.15.183:3300/0 conn(0x7f326006b760 0x7f32600d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265413640 1 -- 172.21.15.50:0/2209444967 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600ce090 con 0x7f326015c570 2023-10-19T10:38:36.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.917+0000 7f3265c14640 1 --2- 172.21.15.50:0/2209444967 >> v2:172.21.15.183:3300/0 conn(0x7f326006b760 0x7f32600d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:36.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3265413640 1 --2- 172.21.15.50:0/2209444967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600d0e30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f32601615b0 tx=0x7f325802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2306c4e299942b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f325ffff640 1 -- 172.21.15.50:0/2209444967 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258004230 con 0x7f326015c570 2023-10-19T10:38:36.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f325ffff640 1 -- 172.21.15.50:0/2209444967 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258004390 con 0x7f326015c570 2023-10-19T10:38:36.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f325ffff640 1 -- 172.21.15.50:0/2209444967 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258003810 con 0x7f326015c570 2023-10-19T10:38:36.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 -- 172.21.15.50:0/2209444967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 msgr2=0x7f32600d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 --2- 172.21.15.50:0/2209444967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600d0e30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f32601615b0 tx=0x7f325802b860 comp rx=0 tx=0).stop 2023-10-19T10:38:36.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 -- 172.21.15.50:0/2209444967 shutdown_connections 2023-10-19T10:38:36.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 --2- 172.21.15.50:0/2209444967 >> v2:172.21.15.183:3300/0 conn(0x7f326006b760 0x7f32600d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 --2- 172.21.15.50:0/2209444967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 --2- 172.21.15.50:0/2209444967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600d0e30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 -- 172.21.15.50:0/2209444967 >> 172.21.15.50:0/2209444967 conn(0x7f3260089ff0 msgr2=0x7f326009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:36.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 -- 172.21.15.50:0/2209444967 shutdown_connections 2023-10-19T10:38:36.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.918+0000 7f3267677640 1 -- 172.21.15.50:0/2209444967 wait complete. 2023-10-19T10:38:36.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.919+0000 7f3267677640 1 Processor -- start 2023-10-19T10:38:36.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 -- start start 2023-10-19T10:38:36.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f326006b760 0x7f32600c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c9e60 con 0x7f326015c570 2023-10-19T10:38:36.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c97e0 con 0x7f326006b760 2023-10-19T10:38:36.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3267677640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c8910 con 0x7f326015c950 2023-10-19T10:38:36.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3264c12640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3264c12640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38340/0 (socket says 172.21.15.50:38340) 2023-10-19T10:38:36.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.920+0000 7f3264c12640 1 -- 172.21.15.50:0/2332613852 learned_addr learned my addr 172.21.15.50:0/2332613852 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:36.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3265c14640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3265413640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f326006b760 0x7f32600c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3264c12640 1 -- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 msgr2=0x7f32600c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3264c12640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3264c12640 1 -- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f326006b760 msgr2=0x7f32600c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:36.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3264c12640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f326006b760 0x7f32600c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:36.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3264c12640 1 -- 172.21.15.50:0/2332613852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258009460 con 0x7f326015c570 2023-10-19T10:38:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3265c14640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3265413640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f326006b760 0x7f32600c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3264c12640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600c6240 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f325400d800 tx=0x7f325400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400b6c0 con 0x7f326015c570 2023-10-19T10:38:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325400b820 con 0x7f326015c570 2023-10-19T10:38:36.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32600c8bf0 con 0x7f326015c570 2023-10-19T10:38:36.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400bbb0 con 0x7f326015c570 2023-10-19T10:38:36.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.921+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32600cc020 con 0x7f326015c570 2023-10-19T10:38:36.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.922+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32600c8de0 con 0x7f326015c570 2023-10-19T10:38:36.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.922+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f325400bd10 con 0x7f326015c570 2023-10-19T10:38:36.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.923+0000 7f325dffb640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f324c06a050 0x7f324c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:36.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.923+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3254088bd0 con 0x7f326015c570 2023-10-19T10:38:36.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.923+0000 7f3265413640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f324c06a050 0x7f324c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:36.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.924+0000 7f3265413640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f324c06a050 0x7f324c06c510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3258034000 tx=0x7f3258008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:36.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:36.927+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3254057c60 con 0x7f326015c570 2023-10-19T10:38:37.142 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.140+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3260073990 con 0x7f326015c570 2023-10-19T10:38:37.143 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.140+0000 7f325dffb640 1 -- 172.21.15.50:0/2332613852 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f32540577f0 con 0x7f326015c570 2023-10-19T10:38:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:37.144 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:38:37.144 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:38:37.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.142+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f324c06a050 msgr2=0x7f324c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.145 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.142+0000 7f3267677640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f324c06a050 0x7f324c06c510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3258034000 tx=0x7f3258008030 comp rx=0 tx=0).stop 2023-10-19T10:38:37.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.142+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 msgr2=0x7f32600c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.142+0000 7f3267677640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600c6240 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f325400d800 tx=0x7f325400dcc0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 shutdown_connections 2023-10-19T10:38:37.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f324c06a050 0x7f324c06c510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.146 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f326015c950 0x7f32600c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f326015c570 0x7f32600c6240 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 --2- 172.21.15.50:0/2332613852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f326006b760 0x7f32600c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 >> 172.21.15.50:0/2332613852 conn(0x7f3260089ff0 msgr2=0x7f32600b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 shutdown_connections 2023-10-19T10:38:37.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.143+0000 7f3267677640 1 -- 172.21.15.50:0/2332613852 wait complete. 2023-10-19T10:38:37.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-19T10:38:27.942428+0000', 'modified': '2023-10-19T10:38:30.587775+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5775}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5775': {'gid': 5775, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6834/1814941523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 1814941523}, {'type': 'v1', 'addr': '172.21.15.50:6834', 'nonce': 1814941523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-19T10:38:37.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-19T10:38:37.559 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-19T10:38:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.669+0000 7f4735628640 1 Processor -- start 2023-10-19T10:38:37.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.669+0000 7f4735628640 1 -- start start 2023-10-19T10:38:37.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f4735628640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f473015a290 0x7f473015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f4735628640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f47300cbb20 0x7f47301609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f4735628640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473015d450 con 0x7f4730160ee0 2023-10-19T10:38:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f4735628640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473006a530 con 0x7f47300cbb20 2023-10-19T10:38:37.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f4735628640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473015a670 con 0x7f473015a290 2023-10-19T10:38:37.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472e7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f47300cbb20 0x7f47301609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f4730160ee0 0x7f47300ccd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57608/0 (socket says 172.21.15.50:57608) 2023-10-19T10:38:37.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472f7fe640 1 -- 172.21.15.50:0/1711375486 learned_addr learned my addr 172.21.15.50:0/1711375486 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:37.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472effd640 1 --2- 172.21.15.50:0/1711375486 >> v2:172.21.15.183:3300/0 conn(0x7f473015a290 0x7f473015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472e7fc640 1 -- 172.21.15.50:0/1711375486 >> v2:172.21.15.183:3300/0 conn(0x7f473015a290 msgr2=0x7f473015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472e7fc640 1 --2- 172.21.15.50:0/1711375486 >> v2:172.21.15.183:3300/0 conn(0x7f473015a290 0x7f473015cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472e7fc640 1 -- 172.21.15.50:0/1711375486 >> v1:172.21.15.50:6789/0 conn(0x7f4730160ee0 legacy=0x7f47300ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472e7fc640 1 -- 172.21.15.50:0/1711375486 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473015d730 con 0x7f47300cbb20 2023-10-19T10:38:37.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472dffb640 1 -- 172.21.15.50:0/1711375486 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76385884 0 0) 0x7f473015d450 con 0x7f4730160ee0 2023-10-19T10:38:37.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.670+0000 7f472effd640 1 --2- 172.21.15.50:0/1711375486 >> v2:172.21.15.183:3300/0 conn(0x7f473015a290 0x7f473015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:37.676 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.671+0000 7f472e7fc640 1 --2- 172.21.15.50:0/1711375486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47300cbb20 0x7f47301609a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47300cf0f0 tx=0x7f471802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5063fff3e8e33197 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.671+0000 7f472dffb640 1 -- 172.21.15.50:0/1711375486 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718031830 con 0x7f47300cbb20 2023-10-19T10:38:37.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.671+0000 7f472dffb640 1 -- 172.21.15.50:0/1711375486 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718031990 con 0x7f47300cbb20 2023-10-19T10:38:37.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.671+0000 7f472dffb640 1 -- 172.21.15.50:0/1711375486 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718031d20 con 0x7f47300cbb20 2023-10-19T10:38:37.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 -- 172.21.15.50:0/1711375486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47300cbb20 msgr2=0x7f47301609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 --2- 172.21.15.50:0/1711375486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47300cbb20 0x7f47301609a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47300cf0f0 tx=0x7f471802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 -- 172.21.15.50:0/1711375486 shutdown_connections 2023-10-19T10:38:37.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 --2- 172.21.15.50:0/1711375486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47300cbb20 0x7f47301609a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 --2- 172.21.15.50:0/1711375486 >> v2:172.21.15.183:3300/0 conn(0x7f473015a290 0x7f473015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 -- 172.21.15.50:0/1711375486 >> 172.21.15.50:0/1711375486 conn(0x7f473006c9a0 msgr2=0x7f473006cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 -- 172.21.15.50:0/1711375486 shutdown_connections 2023-10-19T10:38:37.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.672+0000 7f4735628640 1 -- 172.21.15.50:0/1711375486 wait complete. 2023-10-19T10:38:37.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.673+0000 7f4735628640 1 Processor -- start 2023-10-19T10:38:37.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.673+0000 7f4735628640 1 -- start start 2023-10-19T10:38:37.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f4735628640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47300cbb20 0x7f47300c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f4735628640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 0x7f4730146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f4735628640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4730160ee0 0x7f47300c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f4735628640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300c38a0 con 0x7f473015a290 2023-10-19T10:38:37.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f4735628640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730154db0 con 0x7f47300cbb20 2023-10-19T10:38:37.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f4735628640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4730147410 con 0x7f4730160ee0 2023-10-19T10:38:37.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 0x7f4730146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 0x7f4730146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38352/0 (socket says 172.21.15.50:38352) 2023-10-19T10:38:37.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 -- 172.21.15.50:0/3640551037 learned_addr learned my addr 172.21.15.50:0/3640551037 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472f7fe640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4730160ee0 0x7f47300c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472effd640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47300cbb20 0x7f47300c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 -- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4730160ee0 msgr2=0x7f47300c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4730160ee0 0x7f47300c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 -- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47300cbb20 msgr2=0x7f47300c17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47300cbb20 0x7f47300c17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 -- 172.21.15.50:0/3640551037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718009400 con 0x7f473015a290 2023-10-19T10:38:37.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.674+0000 7f472e7fc640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 0x7f4730146ed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4718000c00 tx=0x7f47180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.675+0000 7f47277fe640 1 -- 172.21.15.50:0/3640551037 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471803aa60 con 0x7f473015a290 2023-10-19T10:38:37.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.675+0000 7f47277fe640 1 -- 172.21.15.50:0/3640551037 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471803abe0 con 0x7f473015a290 2023-10-19T10:38:37.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.675+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47300c2c90 con 0x7f473015a290 2023-10-19T10:38:37.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.675+0000 7f47277fe640 1 -- 172.21.15.50:0/3640551037 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180443f0 con 0x7f473015a290 2023-10-19T10:38:37.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.675+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47300c3250 con 0x7f473015a290 2023-10-19T10:38:37.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.677+0000 7f47277fe640 1 -- 172.21.15.50:0/3640551037 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f471804d450 con 0x7f473015a290 2023-10-19T10:38:37.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.677+0000 7f47277fe640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f471006a050 0x7f471006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.677+0000 7f47277fe640 1 -- 172.21.15.50:0/3640551037 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f47180aa990 con 0x7f473015a290 2023-10-19T10:38:37.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.677+0000 7f472effd640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f471006a050 0x7f471006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.678+0000 7f4738196740 1 Processor -- start 2023-10-19T10:38:37.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.678+0000 7f472effd640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f471006a050 0x7f471006c510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4728008850 tx=0x7f4728008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.678+0000 7f4738196740 1 -- start start 2023-10-19T10:38:37.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f4738196740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 0x55a7fefc10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefc4d50 0x55a7fefc67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7feff0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f4738196740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fef33060 con 0x55a7fefc0cc0 2023-10-19T10:38:37.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fef30750 con 0x55a7fefc4d50 2023-10-19T10:38:37.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7fef2c040 con 0x55a7feff0a90 2023-10-19T10:38:37.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 0x55a7fefc10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 0x55a7fefc10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38354/0 (socket says 172.21.15.50:38354) 2023-10-19T10:38:37.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472effd640 1 -- 172.21.15.50:0/1041278837 learned_addr learned my addr 172.21.15.50:0/1041278837 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472f7fe640 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7feff0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472e7fc640 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefc4d50 0x55a7fefc67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472effd640 1 -- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 msgr2=0x55a7feff0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.679+0000 7f472effd640 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7feff0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f472effd640 1 -- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefc4d50 msgr2=0x55a7fefc67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f472effd640 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefc4d50 0x55a7fefc67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f472effd640 1 -- 172.21.15.50:0/1041278837 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7fef30cb0 con 0x55a7fefc0cc0 2023-10-19T10:38:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f472effd640 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 0x55a7fefc10a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f472800dae0 tx=0x7f4728010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24befb5e5a13748 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f4724ff9640 1 -- 172.21.15.50:0/1041278837 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800b050 con 0x55a7fefc0cc0 2023-10-19T10:38:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f4724ff9640 1 -- 172.21.15.50:0/1041278837 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728017070 con 0x55a7fefc0cc0 2023-10-19T10:38:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f4724ff9640 1 -- 172.21.15.50:0/1041278837 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728012cb0 con 0x55a7fefc0cc0 2023-10-19T10:38:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f4738196740 1 -- 172.21.15.50:0/1041278837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 msgr2=0x55a7fefc10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.680+0000 7f4738196740 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 0x55a7fefc10a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f472800dae0 tx=0x7f4728010000 comp rx=0 tx=0).stop 2023-10-19T10:38:37.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 -- 172.21.15.50:0/1041278837 shutdown_connections 2023-10-19T10:38:37.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7feff0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefc4d50 0x55a7fefc67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 --2- 172.21.15.50:0/1041278837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefc0cc0 0x55a7fefc10a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 -- 172.21.15.50:0/1041278837 >> 172.21.15.50:0/1041278837 conn(0x55a7fefc2d80 msgr2=0x55a7fefc4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 -- 172.21.15.50:0/1041278837 shutdown_connections 2023-10-19T10:38:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 -- 172.21.15.50:0/1041278837 wait complete. 2023-10-19T10:38:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 -- start start 2023-10-19T10:38:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.681+0000 7f4738196740 1 Processor -- start 2023-10-19T10:38:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7feff0a90 0x55a7fefecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7ff00e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff00ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7feee8d00 con 0x55a7fefe29a0 2023-10-19T10:38:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7feee89a0 con 0x55a7feff0a90 2023-10-19T10:38:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7feeae1f0 con 0x55a7fefe25c0 2023-10-19T10:38:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f4738196740 10 client.0 _ll_register_callbacks cb 0x55a7fef23d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-19T10:38:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff00ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff00ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38362/0 (socket says 172.21.15.50:38362) 2023-10-19T10:38:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 -- 172.21.15.50:0/699664322 learned_addr learned my addr 172.21.15.50:0/699664322 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472effd640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7feff0a90 0x55a7fefecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472e7fc640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7ff00e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 -- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 msgr2=0x55a7ff00e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7ff00e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 -- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7feff0a90 msgr2=0x55a7fefecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7feff0a90 0x55a7fefecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.682+0000 7f472f7fe640 1 -- 172.21.15.50:0/699664322 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472800d840 con 0x55a7fefe29a0 2023-10-19T10:38:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f472f7fe640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff00ee70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4720010940 tx=0x7f4720010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f69a5c0045c818e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f470dffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2023-10-19T10:38:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f470dffb640 1 -- 172.21.15.50:0/699664322 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000ebb0 con 0x55a7fefe29a0 2023-10-19T10:38:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f470dffb640 1 -- 172.21.15.50:0/699664322 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f472000ed10 con 0x55a7fefe29a0 2023-10-19T10:38:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7fef00e00 con 0x55a7fefe29a0 2023-10-19T10:38:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f470dffb640 1 -- 172.21.15.50:0/699664322 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f472000eed0 con 0x55a7fefe29a0 2023-10-19T10:38:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 10 client.5976 fetch_fsmap learned FSMap version 15 2023-10-19T10:38:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55a7fef2a6c0 con 0x55a7fefe29a0 2023-10-19T10:38:37.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f470dffb640 1 -- 172.21.15.50:0/699664322 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7fef00e00 con 0x55a7fefe29a0 2023-10-19T10:38:37.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f470dffb640 1 -- 172.21.15.50:0/699664322 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4720012040 con 0x55a7fefe29a0 2023-10-19T10:38:37.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 10 client.5976 fetch_fsmap finished waiting for FSMap version 15 2023-10-19T10:38:37.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 10 client.5976 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-19T10:38:37.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 20 client.5976 populate_metadata read hostname 'smithi050' 2023-10-19T10:38:37.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7feff23e0 0x55a7feff27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 4 client.5976 mds_command: new command op to 5775 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-19T10:38:37.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 --> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7fef25020 con 0x55a7feff23e0 2023-10-19T10:38:37.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.683+0000 7f472effd640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7feff23e0 0x55a7feff27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.684+0000 7f470dffb640 10 client.5976 ms_handle_connect on v2:172.21.15.50:6832/1814941523 2023-10-19T10:38:37.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.684+0000 7f472effd640 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7feff23e0 0x55a7feff27c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb8ba03089891a server_cookie=33875d7aa9f76d10 in_seq=0 out_seq=0 2023-10-19T10:38:37.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.684+0000 7f470dffb640 1 -- 172.21.15.50:0/699664322 <== mds.0 v2:172.21.15.50:6832/1814941523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7fef25020 con 0x55a7feff23e0 2023-10-19T10:38:37.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.684+0000 7f470dffb640 10 client.5976 handle_command_reply: tid=0 2023-10-19T10:38:37.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.684+0000 7f4738196740 1 client.5976 shutdown 2023-10-19T10:38:37.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 msgr2=0x55a7ff00ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff00ee70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4720010940 tx=0x7f4720010d40 comp rx=0 tx=0).stop 2023-10-19T10:38:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 shutdown_connections 2023-10-19T10:38:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff00ee70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7feff23e0 0x55a7feff27c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7ff00e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 --2- 172.21.15.50:0/699664322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7feff0a90 0x55a7fefecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f470dffb640 0 client.5976 ms_handle_reset on v2:172.21.15.50:6832/1814941523 2023-10-19T10:38:37.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 >> 172.21.15.50:0/699664322 conn(0x55a7fefc0cc0 msgr2=0x55a7ff00cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 shutdown_connections 2023-10-19T10:38:37.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 1 -- 172.21.15.50:0/699664322 wait complete. 2023-10-19T10:38:37.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.685+0000 7f4738196740 20 client.5976 trim_cache size 0 max 16384 2023-10-19T10:38:37.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.700+0000 7f4738196740 1 Processor -- start 2023-10-19T10:38:37.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.700+0000 7f4738196740 1 -- start start 2023-10-19T10:38:37.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe25c0 0x55a7fefc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f4738196740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff01b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7fefc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f4738196740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fefc1af0 con 0x55a7fefe29a0 2023-10-19T10:38:37.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fefddb10 con 0x55a7fefe25c0 2023-10-19T10:38:37.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7fefc5de0 con 0x55a7feff0a90 2023-10-19T10:38:37.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff01b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff01b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38378/0 (socket says 172.21.15.50:38378) 2023-10-19T10:38:37.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 -- 172.21.15.50:0/1974713319 learned_addr learned my addr 172.21.15.50:0/1974713319 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 -- 172.21.15.50:0/1974713319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 msgr2=0x55a7fefc6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7fefc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 -- 172.21.15.50:0/1974713319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe25c0 msgr2=0x55a7fefc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe25c0 0x55a7fefc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 -- 172.21.15.50:0/1974713319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7feecac20 con 0x55a7fefe29a0 2023-10-19T10:38:37.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f472e7fc640 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff01b720 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f47180798c0 tx=0x7f471807f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458a0ea53f67f89f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f47257fa640 1 -- 172.21.15.50:0/1974713319 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180762d0 con 0x55a7fefe29a0 2023-10-19T10:38:37.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f47257fa640 1 -- 172.21.15.50:0/1974713319 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718076430 con 0x55a7fefe29a0 2023-10-19T10:38:37.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.701+0000 7f47257fa640 1 -- 172.21.15.50:0/1974713319 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180792f0 con 0x55a7fefe29a0 2023-10-19T10:38:37.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 -- 172.21.15.50:0/1974713319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 msgr2=0x55a7ff01b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff01b720 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f47180798c0 tx=0x7f471807f040 comp rx=0 tx=0).stop 2023-10-19T10:38:37.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 -- 172.21.15.50:0/1974713319 shutdown_connections 2023-10-19T10:38:37.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7feff0a90 0x55a7fefc6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7fefe29a0 0x55a7ff01b720 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 --2- 172.21.15.50:0/1974713319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe25c0 0x55a7fefc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 -- 172.21.15.50:0/1974713319 >> 172.21.15.50:0/1974713319 conn(0x55a7fefc0cc0 msgr2=0x55a7fefecab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 -- 172.21.15.50:0/1974713319 shutdown_connections 2023-10-19T10:38:37.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 -- 172.21.15.50:0/1974713319 wait complete. 2023-10-19T10:38:37.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 -- start start 2023-10-19T10:38:37.721 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.702+0000 7f4738196740 1 Processor -- start 2023-10-19T10:38:37.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7feff0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe29a0 0x55a7feff13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 0x55a7ff025340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ff075ec0 con 0x55a7feff0a90 2023-10-19T10:38:37.722 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ff01e1c0 con 0x55a7fefe29a0 2023-10-19T10:38:37.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7feff38f0 con 0x55a7fefe25c0 2023-10-19T10:38:37.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f4738196740 10 client.0 _ll_register_callbacks cb 0x7f4728008f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-19T10:38:37.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f472f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 0x55a7ff025340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f472f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 0x55a7ff025340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38380/0 (socket says 172.21.15.50:38380) 2023-10-19T10:38:37.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f472f7fe640 1 -- 172.21.15.50:0/2865086550 learned_addr learned my addr 172.21.15.50:0/2865086550 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f472effd640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7feff0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.703+0000 7f472e7fc640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe29a0 0x55a7feff13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472f7fe640 1 -- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 msgr2=0x55a7feff0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472f7fe640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7feff0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472f7fe640 1 -- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe29a0 msgr2=0x55a7feff13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472f7fe640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe29a0 0x55a7feff13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472f7fe640 1 -- 172.21.15.50:0/2865086550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47300c3250 con 0x55a7feff0a90 2023-10-19T10:38:37.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472e7fc640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe29a0 0x55a7feff13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:37.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f472f7fe640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 0x55a7ff025340 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4720010940 tx=0x7f4720001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da0108118b77a63 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f470dffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2023-10-19T10:38:37.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f470dffb640 1 -- 172.21.15.50:0/2865086550 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720010500 con 0x55a7feff0a90 2023-10-19T10:38:37.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f470dffb640 1 -- 172.21.15.50:0/2865086550 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4720001c80 con 0x55a7feff0a90 2023-10-19T10:38:37.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4728017070 con 0x55a7feff0a90 2023-10-19T10:38:37.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f470dffb640 1 -- 172.21.15.50:0/2865086550 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f472000e590 con 0x55a7feff0a90 2023-10-19T10:38:37.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 10 client.5982 fetch_fsmap learned FSMap version 15 2023-10-19T10:38:37.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55a7feef01f0 con 0x55a7feff0a90 2023-10-19T10:38:37.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f470dffb640 1 -- 172.21.15.50:0/2865086550 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728017070 con 0x55a7feff0a90 2023-10-19T10:38:37.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f470dffb640 1 -- 172.21.15.50:0/2865086550 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4720012050 con 0x55a7feff0a90 2023-10-19T10:38:37.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 10 client.5982 fetch_fsmap finished waiting for FSMap version 15 2023-10-19T10:38:37.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 10 client.5982 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-19T10:38:37.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 20 client.5982 populate_metadata read hostname 'smithi050' 2023-10-19T10:38:37.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7ff025970 0x55a7ff025d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 4 client.5982 mds_command: new command op to 5775 tid=0[{"prefix": "status"}] 2023-10-19T10:38:37.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.704+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 --> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a7fef24d40 con 0x55a7ff025970 2023-10-19T10:38:37.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f472effd640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7ff025970 0x55a7ff025d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f470dffb640 10 client.5982 ms_handle_connect on v2:172.21.15.50:6832/1814941523 2023-10-19T10:38:37.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f472effd640 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7ff025970 0x55a7ff025d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7312dea06b623a2d server_cookie=6ac52ee64400f63f in_seq=0 out_seq=0 2023-10-19T10:38:37.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f470dffb640 1 -- 172.21.15.50:0/2865086550 <== mds.0 v2:172.21.15.50:6832/1814941523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55a7fef24d40 con 0x55a7ff025970 2023-10-19T10:38:37.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f470dffb640 10 client.5982 handle_command_reply: tid=0 2023-10-19T10:38:37.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 client.5982 shutdown 2023-10-19T10:38:37.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 msgr2=0x55a7ff025340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 0x55a7ff025340 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4720010940 tx=0x7f4720001250 comp rx=0 tx=0).stop 2023-10-19T10:38:37.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 shutdown_connections 2023-10-19T10:38:37.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a7feff0a90 0x55a7ff025340 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fefe29a0 0x55a7feff13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.50:6832/1814941523,v1:172.21.15.50:6834/1814941523] conn(0x55a7ff025970 0x55a7ff025d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f470dffb640 0 client.5982 ms_handle_reset on v2:172.21.15.50:6832/1814941523 2023-10-19T10:38:37.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 --2- 172.21.15.50:0/2865086550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55a7fefe25c0 0x55a7feff0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 >> 172.21.15.50:0/2865086550 conn(0x55a7fefc0cc0 msgr2=0x55a7fefc6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 shutdown_connections 2023-10-19T10:38:37.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 1 -- 172.21.15.50:0/2865086550 wait complete. 2023-10-19T10:38:37.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.705+0000 7f4738196740 20 client.5982 trim_cache size 0 max 16384 2023-10-19T10:38:37.732 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:38:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "2b47c9ed-f967-49ff-9d46-d95ee8c39a83", 2023-10-19T10:38:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2023-10-19T10:38:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5775, 2023-10-19T10:38:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2023-10-19T10:38:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2023-10-19T10:38:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2023-10-19T10:38:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 9.1233475730000002, 2023-10-19T10:38:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 15, 2023-10-19T10:38:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 44, 2023-10-19T10:38:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 42, 2023-10-19T10:38:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 16.99625722 2023-10-19T10:38:37.734 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:38:37.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f471006a050 msgr2=0x7f471006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f471006a050 0x7f471006c510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4728008850 tx=0x7f4728008040 comp rx=0 tx=0).stop 2023-10-19T10:38:37.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 msgr2=0x7f4730146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 0x7f4730146ed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4718000c00 tx=0x7f47180046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 shutdown_connections 2023-10-19T10:38:37.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f471006a050 0x7f471006c510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4730160ee0 0x7f47300c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f473015a290 0x7f4730146ed0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 --2- 172.21.15.50:0/3640551037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47300cbb20 0x7f47300c17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 >> 172.21.15.50:0/3640551037 conn(0x7f473006c9a0 msgr2=0x7f473015dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 shutdown_connections 2023-10-19T10:38:37.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.706+0000 7f4735628640 1 -- 172.21.15.50:0/3640551037 wait complete. 2023-10-19T10:38:37.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:38:37.849 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 Processor -- start 2023-10-19T10:38:37.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 -- start start 2023-10-19T10:38:37.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f751415c570 0x7f75140d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f751415c950 0x7f75140cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75140cde60 con 0x7f751406b760 2023-10-19T10:38:37.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751406aa30 con 0x7f751415c950 2023-10-19T10:38:37.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.843+0000 7f7519c9d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751406b2d0 con 0x7f751415c570 2023-10-19T10:38:37.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f751415c950 0x7f75140cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f751415c950 0x7f75140cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38390/0 (socket says 172.21.15.50:38390) 2023-10-19T10:38:37.851 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 -- 172.21.15.50:0/1494970329 learned_addr learned my addr 172.21.15.50:0/1494970329 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f75137fe640 1 --2- 172.21.15.50:0/1494970329 >> v2:172.21.15.183:3300/0 conn(0x7f751415c570 0x7f75140d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 -- 172.21.15.50:0/1494970329 >> v2:172.21.15.183:3300/0 conn(0x7f751415c570 msgr2=0x7f75140d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:37.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 --2- 172.21.15.50:0/1494970329 >> v2:172.21.15.183:3300/0 conn(0x7f751415c570 0x7f75140d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 -- 172.21.15.50:0/1494970329 >> v1:172.21.15.183:6789/0 conn(0x7f751406b760 legacy=0x7f75140d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f7512ffd640 1 -- 172.21.15.50:0/1494970329 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75140ce090 con 0x7f751415c950 2023-10-19T10:38:37.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.844+0000 7f75137fe640 1 --2- 172.21.15.50:0/1494970329 >> v2:172.21.15.183:3300/0 conn(0x7f751415c570 0x7f75140d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:37.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7512ffd640 1 --2- 172.21.15.50:0/1494970329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751415c950 0x7f75140cd920 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75140d2670 tx=0x7f750802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a50c475cfe23d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f75127fc640 1 -- 172.21.15.50:0/1494970329 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508031870 con 0x7f751415c950 2023-10-19T10:38:37.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f75127fc640 1 -- 172.21.15.50:0/1494970329 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75080319d0 con 0x7f751415c950 2023-10-19T10:38:37.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f75127fc640 1 -- 172.21.15.50:0/1494970329 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508031d60 con 0x7f751415c950 2023-10-19T10:38:37.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 -- 172.21.15.50:0/1494970329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751415c950 msgr2=0x7f75140cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 --2- 172.21.15.50:0/1494970329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751415c950 0x7f75140cd920 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75140d2670 tx=0x7f750802b910 comp rx=0 tx=0).stop 2023-10-19T10:38:37.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 -- 172.21.15.50:0/1494970329 shutdown_connections 2023-10-19T10:38:37.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 --2- 172.21.15.50:0/1494970329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751415c950 0x7f75140cd920 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 --2- 172.21.15.50:0/1494970329 >> v2:172.21.15.183:3300/0 conn(0x7f751415c570 0x7f75140d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 -- 172.21.15.50:0/1494970329 >> 172.21.15.50:0/1494970329 conn(0x7f7514089ff0 msgr2=0x7f751409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:37.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 -- 172.21.15.50:0/1494970329 shutdown_connections 2023-10-19T10:38:37.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.845+0000 7f7519c9d640 1 -- 172.21.15.50:0/1494970329 wait complete. 2023-10-19T10:38:37.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.847+0000 7f7519c9d640 1 Processor -- start 2023-10-19T10:38:37.855 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.847+0000 7f7519c9d640 1 -- start start 2023-10-19T10:38:37.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.847+0000 7f7519c9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 0x7f751415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.847+0000 7f7519c9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751415c570 0x7f7514157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7519c9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f751415c950 0x7f75140c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7519c9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75140c6350 con 0x7f751406b760 2023-10-19T10:38:37.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7519c9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751415e280 con 0x7f751415c570 2023-10-19T10:38:37.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7519c9d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751415e560 con 0x7f751415c950 2023-10-19T10:38:37.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 0x7f751415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 0x7f751415d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38402/0 (socket says 172.21.15.50:38402) 2023-10-19T10:38:37.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 -- 172.21.15.50:0/2873404463 learned_addr learned my addr 172.21.15.50:0/2873404463 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:37.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7513fff640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f751415c950 0x7f75140c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7512ffd640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751415c570 0x7f7514157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 -- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f751415c950 msgr2=0x7f75140c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f751415c950 0x7f75140c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 -- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751415c570 msgr2=0x7f7514157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751415c570 0x7f7514157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 -- 172.21.15.50:0/2873404463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75080094d0 con 0x7f751406b760 2023-10-19T10:38:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f75137fe640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 0x7f751415d980 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f750400e810 tx=0x7f750400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504004030 con 0x7f751406b760 2023-10-19T10:38:37.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504004190 con 0x7f751406b760 2023-10-19T10:38:37.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504005990 con 0x7f751406b760 2023-10-19T10:38:37.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.848+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751415e840 con 0x7f751406b760 2023-10-19T10:38:37.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.849+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751415ee00 con 0x7f751406b760 2023-10-19T10:38:37.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.849+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514073990 con 0x7f751406b760 2023-10-19T10:38:37.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.850+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7504005af0 con 0x7f751406b760 2023-10-19T10:38:37.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.850+0000 7f74fbfff640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750006a050 0x7f750006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:37.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.850+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7504088560 con 0x7f751406b760 2023-10-19T10:38:37.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.851+0000 7f7512ffd640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750006a050 0x7f750006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:37.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.851+0000 7f7512ffd640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750006a050 0x7f750006c510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7508036000 tx=0x7f750803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:37.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:37.855+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7504053b80 con 0x7f751406b760 2023-10-19T10:38:38.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.065+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f75140d3480 con 0x7f751406b760 2023-10-19T10:38:38.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.065+0000 7f74fbfff640 1 -- 172.21.15.50:0/2873404463 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f75040571a0 con 0x7f751406b760 2023-10-19T10:38:38.068 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-19T10:38:38.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750006a050 msgr2=0x7f750006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:38.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750006a050 0x7f750006c510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7508036000 tx=0x7f750803c040 comp rx=0 tx=0).stop 2023-10-19T10:38:38.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 msgr2=0x7f751415d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:38.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 0x7f751415d980 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f750400e810 tx=0x7f750400ecd0 comp rx=0 tx=0).stop 2023-10-19T10:38:38.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 shutdown_connections 2023-10-19T10:38:38.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750006a050 0x7f750006c510 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:38.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f751415c950 0x7f75140c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:38.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751415c570 0x7f7514157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:38.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 --2- 172.21.15.50:0/2873404463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f751406b760 0x7f751415d980 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:38.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 >> 172.21.15.50:0/2873404463 conn(0x7f7514089ff0 msgr2=0x7f75140cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:38.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 shutdown_connections 2023-10-19T10:38:38.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:38.067+0000 7f7519c9d640 1 -- 172.21.15.50:0/2873404463 wait complete. 2023-10-19T10:38:38.080 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:38:38.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:38:38.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-19T10:38:38.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-19T10:38:38.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-19T10:38:38.082 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:38:38.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:38:38.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:38:38.083 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:38:38.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.084 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-19T10:38:38.100 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:38:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:38:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:38:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:38:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.101 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:38:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2023-10-19T10:38:38.102 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:38:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2023-10-19T10:38:38.103 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:38:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2023-10-19T10:38:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-19T10:38:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42583sec preferred_lft 42583sec 2023-10-19T10:38:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2023-10-19T10:38:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.104 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:38:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2023-10-19T10:38:38.105 INFO:teuthology.orchestra.run.smithi050.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:38:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c2:88:d7:94:a0:7a brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-19T10:38:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a497:7bff:fe82:fcde/64 scope link 2023-10-19T10:38:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.106 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:38:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether fa:5b:82:63:89:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-19T10:38:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::f85b:82ff:fe63:89b9/64 scope link 2023-10-19T10:38:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.107 INFO:teuthology.orchestra.run.smithi050.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:38:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c2:88:d7:94:a0:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-19T10:38:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c088:d7ff:fe94:a07a/64 scope link 2023-10-19T10:38:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:38.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.108 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:38:38.170 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-19T10:38:38.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.172 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-19T10:38:38.227 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-19T10:38:38.228 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:38:38.228 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-19T10:38:38.228 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-19T10:38:38.228 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:38:38.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.229 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:38:38.230 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:38:38.230 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-19T10:38:38.230 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-19T10:38:38.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-19T10:38:38.231 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:38:38.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.335 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:38:38.350 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-19T10:38:38.350 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-19T10:38:38.351 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:38:38.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.352 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2023-10-19T10:38:38.421 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-19T10:38:38.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:38:38.422 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-19T10:38:38.476 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:38:38.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:38:38.477 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-19T10:38:38.531 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:38:38.602 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:38:38.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:38:38.673 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:38:38.674 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:38:38.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.674 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:38:38.729 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-19T10:38:38.730 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-19T10:38:38.772 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:38:38.803 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:38:38.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:38:38.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:2023-10-19T10:38:38.863+0000 7f19b9636480 -1 init, newargv = 0x560167ce0ce0 newargc=15 2023-10-19T10:38:38.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[105851]: starting ceph client 2023-10-19T10:38:38.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[105851]: starting fuse 2023-10-19T10:38:38.875 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:38:38.876 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:38:38.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:38.878 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:38:38.932 INFO:teuthology.orchestra.run.smithi050.stdout:41 2023-10-19T10:38:38.933 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-19T10:38:38.933 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:38:38.934 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-19T10:38:38.934 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-19T10:38:38.935 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-19T10:38:38.935 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-19T10:38:38.936 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:38:38.936 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-19T10:38:38.936 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-19T10:38:38.937 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-19T10:38:38.937 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-19T10:38:38.938 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:38:38.938 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-19T10:38:38.938 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-19T10:38:38.939 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-19T10:38:38.939 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-19T10:38:38.940 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:38:38.940 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-19T10:38:38.940 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:38:38.941 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:38:38.941 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:38:38.942 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-19T10:38:38.943 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-19T10:38:38.943 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-19T10:38:38.943 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.0")) 2023-10-19T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-19T10:38:39.057 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.0.105851.asok 2023-10-19T10:38:39.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:38:39.068 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105851.asok 2023-10-19T10:38:39.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.069 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105851.asok status 2023-10-19T10:38:39.211 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:38:39.211 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-19T10:38:39.211 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:38:39.211 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:38:39.211 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "0", 2023-10-19T10:38:39.212 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-19T10:38:39.212 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-19T10:38:39.212 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "105851", 2023-10-19T10:38:39.212 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-19T10:38:39.212 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:39.213 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-19T10:38:39.213 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-19T10:38:39.213 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5994, 2023-10-19T10:38:39.213 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-19T10:38:39.213 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-19T10:38:39.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-19T10:38:39.214 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 5994 2023-10-19T10:38:39.214 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:39.214 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:38:39.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:38:39.215 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:38:39.215 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 216420809 2023-10-19T10:38:39.215 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:38:39.215 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:39.215 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:38:39.215 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:38:39.216 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:38:39.216 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 216420809 2023-10-19T10:38:39.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:38:39.216 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.5994 192.168.0.1:0/216420809", 2023-10-19T10:38:39.216 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/216420809", 2023-10-19T10:38:39.217 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-19T10:38:39.217 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 15, 2023-10-19T10:38:39.217 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 44, 2023-10-19T10:38:39.217 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:38:39.217 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-19T10:38:39.218 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-19T10:38:39.218 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:38:39.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.219 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-19T10:38:39.220 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-19T10:38:39.235 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-19T10:38:39.236 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-19T10:38:39.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.236 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-19T10:38:39.306 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:38:39.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.450+0000 7fa7b401c640 1 Processor -- start 2023-10-19T10:38:39.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.450+0000 7fa7b401c640 1 -- start start 2023-10-19T10:38:39.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b401c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa7ac15c570 0x7fa7ac0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:39.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b401c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa7ac0cd920 0x7fa7ac0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:39.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b401c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ac06aa30 con 0x7fa7ac15c950 2023-10-19T10:38:39.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b401c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac06b2d0 con 0x7fa7ac15c570 2023-10-19T10:38:39.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b401c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac0ce090 con 0x7fa7ac0cd920 2023-10-19T10:38:39.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b1db8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa7ac15c570 0x7fa7ac0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:39.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b15b7640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa7ac15c950 0x7fa7ac06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57618/0 (socket says 172.21.15.50:57618) 2023-10-19T10:38:39.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b1db8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa7ac15c570 0x7fa7ac0d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38414/0 (socket says 172.21.15.50:38414) 2023-10-19T10:38:39.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b15b7640 1 -- 172.21.15.50:0/1951624593 learned_addr learned my addr 172.21.15.50:0/1951624593 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:39.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b25b9640 1 --2- 172.21.15.50:0/1951624593 >> v2:172.21.15.183:3300/0 conn(0x7fa7ac0cd920 0x7fa7ac0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:39.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b1db8640 1 -- 172.21.15.50:0/1951624593 >> v2:172.21.15.183:3300/0 conn(0x7fa7ac0cd920 msgr2=0x7fa7ac0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:39.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b1db8640 1 --2- 172.21.15.50:0/1951624593 >> v2:172.21.15.183:3300/0 conn(0x7fa7ac0cd920 0x7fa7ac0d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b1db8640 1 -- 172.21.15.50:0/1951624593 >> v1:172.21.15.50:6789/0 conn(0x7fa7ac15c950 legacy=0x7fa7ac06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:39.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b0db6640 1 -- 172.21.15.50:0/1951624593 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325713158 0 0) 0x7fa7ac06aa30 con 0x7fa7ac15c950 2023-10-19T10:38:39.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b1db8640 1 -- 172.21.15.50:0/1951624593 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac06ac80 con 0x7fa7ac15c570 2023-10-19T10:38:39.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.451+0000 7fa7b25b9640 1 --2- 172.21.15.50:0/1951624593 >> v2:172.21.15.183:3300/0 conn(0x7fa7ac0cd920 0x7fa7ac0d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:39.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.452+0000 7fa7b1db8640 1 --2- 172.21.15.50:0/1951624593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac15c570 0x7fa7ac0d0e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0d3770 tx=0x7fa7a802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc20856185c25bd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:39.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.452+0000 7fa7b0db6640 1 -- 172.21.15.50:0/1951624593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a802e070 con 0x7fa7ac15c570 2023-10-19T10:38:39.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.452+0000 7fa7b0db6640 1 -- 172.21.15.50:0/1951624593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80315a0 con 0x7fa7ac15c570 2023-10-19T10:38:39.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.452+0000 7fa7b0db6640 1 -- 172.21.15.50:0/1951624593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8031930 con 0x7fa7ac15c570 2023-10-19T10:38:39.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 -- 172.21.15.50:0/1951624593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac15c570 msgr2=0x7fa7ac0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:39.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 --2- 172.21.15.50:0/1951624593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac15c570 0x7fa7ac0d0e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0d3770 tx=0x7fa7a802b840 comp rx=0 tx=0).stop 2023-10-19T10:38:39.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 -- 172.21.15.50:0/1951624593 shutdown_connections 2023-10-19T10:38:39.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 --2- 172.21.15.50:0/1951624593 >> v2:172.21.15.183:3300/0 conn(0x7fa7ac0cd920 0x7fa7ac0d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 --2- 172.21.15.50:0/1951624593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac15c570 0x7fa7ac0d0e30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 -- 172.21.15.50:0/1951624593 >> 172.21.15.50:0/1951624593 conn(0x7fa7ac089ff0 msgr2=0x7fa7ac09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:39.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 -- 172.21.15.50:0/1951624593 shutdown_connections 2023-10-19T10:38:39.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.453+0000 7fa7b401c640 1 -- 172.21.15.50:0/1951624593 wait complete. 2023-10-19T10:38:39.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.454+0000 7fa7b401c640 1 Processor -- start 2023-10-19T10:38:39.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.454+0000 7fa7b401c640 1 -- start start 2023-10-19T10:38:39.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 0x7fa7ac15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7ac15c570 0x7fa7ac1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7ac15c950 0x7fa7ac15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac15aa90 con 0x7fa7ac0cd920 2023-10-19T10:38:39.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1f25a0 con 0x7fa7ac15c950 2023-10-19T10:38:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1f2880 con 0x7fa7ac15c570 2023-10-19T10:38:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 0x7fa7ac15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 0x7fa7ac15a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38416/0 (socket says 172.21.15.50:38416) 2023-10-19T10:38:39.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 -- 172.21.15.50:0/2331860260 learned_addr learned my addr 172.21.15.50:0/2331860260 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:39.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b25b9640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7ac15c950 0x7fa7ac15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:39.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b15b7640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7ac15c570 0x7fa7ac1f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:39.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 -- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7ac15c570 msgr2=0x7fa7ac1f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:39.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7ac15c570 0x7fa7ac1f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 -- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7ac15c950 msgr2=0x7fa7ac15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:39.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7ac15c950 0x7fa7ac15e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 -- 172.21.15.50:0/2331860260 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80093a0 con 0x7fa7ac0cd920 2023-10-19T10:38:39.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b25b9640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7ac15c950 0x7fa7ac15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:39.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b1db8640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 0x7fa7ac15a550 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:39.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8004230 con 0x7fa7ac0cd920 2023-10-19T10:38:39.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80043d0 con 0x7fa7ac0cd920 2023-10-19T10:38:39.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8005bd0 con 0x7fa7ac0cd920 2023-10-19T10:38:39.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac1f2b60 con 0x7fa7ac0cd920 2023-10-19T10:38:39.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.455+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac15e2e0 con 0x7fa7ac0cd920 2023-10-19T10:38:39.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.456+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac073990 con 0x7fa7ac0cd920 2023-10-19T10:38:39.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.457+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa7a8005d30 con 0x7fa7ac0cd920 2023-10-19T10:38:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.457+0000 7fa7a27fc640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa79006a050 0x7fa79006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.457+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa7a80aab00 con 0x7fa7ac0cd920 2023-10-19T10:38:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.458+0000 7fa7b15b7640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa79006a050 0x7fa79006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.458+0000 7fa7b15b7640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa79006a050 0x7fa79006c510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0b8360 tx=0x7fa79c007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.462+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa7a8076220 con 0x7fa7ac0cd920 2023-10-19T10:38:39.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.677+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7ac15e4d0 con 0x7fa7ac0cd920 2023-10-19T10:38:39.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.677+0000 7fa7a27fc640 1 -- 172.21.15.50:0/2331860260 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7a804b070 con 0x7fa7ac0cd920 2023-10-19T10:38:39.680 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-19T10:38:39.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa79006a050 msgr2=0x7fa79006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:39.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa79006a050 0x7fa79006c510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0b8360 tx=0x7fa79c007b50 comp rx=0 tx=0).stop 2023-10-19T10:38:39.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 msgr2=0x7fa7ac15a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:39.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 0x7fa7ac15a550 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a80046c0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 shutdown_connections 2023-10-19T10:38:39.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa79006a050 0x7fa79006c510 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7ac15c950 0x7fa7ac15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7ac15c570 0x7fa7ac1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 --2- 172.21.15.50:0/2331860260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7ac0cd920 0x7fa7ac15a550 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:39.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 >> 172.21.15.50:0/2331860260 conn(0x7fa7ac089ff0 msgr2=0x7fa7ac0b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:39.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 shutdown_connections 2023-10-19T10:38:39.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:39.679+0000 7fa7b401c640 1 -- 172.21.15.50:0/2331860260 wait complete. 2023-10-19T10:38:39.691 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:38:39.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:38:39.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-10-19T10:38:39.693 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-19T10:38:39.693 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:39.693 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:38:39.693 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:38:39.694 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:38:39.694 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-19T10:38:39.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.695 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-10-19T10:38:39.714 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:38:39.714 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:38:39.714 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:38:39.715 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.715 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-10-19T10:38:39.715 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.715 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:38:39.715 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:39.716 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f0 2023-10-19T10:38:39.716 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:38:39.716 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:39.716 INFO:teuthology.orchestra.run.smithi183.stdout: altname enp7s0f1 2023-10-19T10:38:39.716 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:38:39.717 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:39.717 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:38:39.717 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:39.717 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-19T10:38:39.717 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42589sec preferred_lft 42589sec 2023-10-19T10:38:39.717 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::ec4:7aff:fe88:8085/64 scope link 2023-10-19T10:38:39.718 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.718 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:38:39.718 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether ea:e2:d8:35:f6:0f brd ff:ff:ff:ff:ff:ff 2023-10-19T10:38:39.718 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-19T10:38:39.718 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.719 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc40:d8ff:fe0c:e190/64 scope link 2023-10-19T10:38:39.719 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.719 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:38:39.719 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether ea:e2:d8:35:f6:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-19T10:38:39.719 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::e8e2:d8ff:fe35:f60f/64 scope link 2023-10-19T10:38:39.720 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.720 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:38:39.720 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether f2:53:81:73:61:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-19T10:38:39.720 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::f053:81ff:fe73:618b/64 scope link 2023-10-19T10:38:39.720 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:38:39.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.721 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:38:39.784 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-10-19T10:38:39.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.786 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-10-19T10:38:39.844 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-10-19T10:38:39.844 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:38:39.844 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-19T10:38:39.844 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-19T10:38:39.845 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:38:39.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.845 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-10-19T10:38:39.846 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-10-19T10:38:39.846 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-19T10:38:39.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-19T10:38:39.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-19T10:38:39.847 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-10-19T10:38:39.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.945 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-10-19T10:38:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-19T10:38:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-19T10:38:39.961 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-19T10:38:39.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:39.961 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-10-19T10:38:40.027 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-19T10:38:40.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:38:40.028 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:40.082 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-19T10:38:40.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:38:40.083 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:40.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2023-10-19T10:38:40.205 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:38:40.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:38:40.271 INFO:teuthology.orchestra.run.smithi183.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:38:40.272 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:38:40.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:40.273 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2023-10-19T10:38:40.328 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-19T10:38:40.328 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-19T10:38:40.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo modprobe fuse 2023-10-19T10:38:40.397 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:38:40.397 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:38:40.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:2023-10-19T10:38:40.452+0000 7f5880c0b480 -1 init, newargv = 0x559ce24a3860 newargc=15 2023-10-19T10:38:40.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[95663]: starting ceph client 2023-10-19T10:38:40.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[95663]: starting fuse 2023-10-19T10:38:40.465 INFO:teuthology.orchestra.run.smithi183.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:38:40.466 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:38:40.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:40.468 DEBUG:teuthology.orchestra.run.smithi183:> ls /sys/fs/fuse/connections 2023-10-19T10:38:40.522 INFO:teuthology.orchestra.run.smithi183.stdout:41 2023-10-19T10:38:40.522 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-19T10:38:40.523 DEBUG:teuthology.orchestra.run.smithi183:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:38:40.523 DEBUG:teuthology.orchestra.run.smithi183:> import glob 2023-10-19T10:38:40.524 DEBUG:teuthology.orchestra.run.smithi183:> import re 2023-10-19T10:38:40.524 DEBUG:teuthology.orchestra.run.smithi183:> import os 2023-10-19T10:38:40.524 DEBUG:teuthology.orchestra.run.smithi183:> import subprocess 2023-10-19T10:38:40.525 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:38:40.525 DEBUG:teuthology.orchestra.run.smithi183:> def _find_admin_socket(client_name): 2023-10-19T10:38:40.526 DEBUG:teuthology.orchestra.run.smithi183:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-19T10:38:40.526 DEBUG:teuthology.orchestra.run.smithi183:> files = glob.glob(asok_path) 2023-10-19T10:38:40.527 DEBUG:teuthology.orchestra.run.smithi183:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-19T10:38:40.527 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:38:40.527 DEBUG:teuthology.orchestra.run.smithi183:> # Given a non-glob path, it better be there 2023-10-19T10:38:40.528 DEBUG:teuthology.orchestra.run.smithi183:> if "*" not in asok_path: 2023-10-19T10:38:40.528 DEBUG:teuthology.orchestra.run.smithi183:> assert(len(files) == 1) 2023-10-19T10:38:40.528 DEBUG:teuthology.orchestra.run.smithi183:> return files[0] 2023-10-19T10:38:40.529 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:38:40.529 DEBUG:teuthology.orchestra.run.smithi183:> for f in files: 2023-10-19T10:38:40.530 DEBUG:teuthology.orchestra.run.smithi183:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:38:40.530 DEBUG:teuthology.orchestra.run.smithi183:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:38:40.530 DEBUG:teuthology.orchestra.run.smithi183:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:38:40.531 DEBUG:teuthology.orchestra.run.smithi183:> contents = proc_f.read() 2023-10-19T10:38:40.531 DEBUG:teuthology.orchestra.run.smithi183:> if mountpoint in contents: 2023-10-19T10:38:40.531 DEBUG:teuthology.orchestra.run.smithi183:> return f 2023-10-19T10:38:40.532 DEBUG:teuthology.orchestra.run.smithi183:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:38:40.532 DEBUG:teuthology.orchestra.run.smithi183:> 2023-10-19T10:38:40.533 DEBUG:teuthology.orchestra.run.smithi183:> print(_find_admin_socket("client.1")) 2023-10-19T10:38:40.533 DEBUG:teuthology.orchestra.run.smithi183:> ' 2023-10-19T10:38:40.641 INFO:teuthology.orchestra.run.smithi183.stdout:/var/run/ceph/ceph-client.1.95663.asok 2023-10-19T10:38:40.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-19T10:38:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:38:40.651 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95663.asok 2023-10-19T10:38:40.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:40.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95663.asok status 2023-10-19T10:38:40.786 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-10-19T10:38:40.786 INFO:teuthology.orchestra.run.smithi183.stdout: "metadata": { 2023-10-19T10:38:40.786 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:38:40.786 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:38:40.787 INFO:teuthology.orchestra.run.smithi183.stdout: "entity_id": "1", 2023-10-19T10:38:40.787 INFO:teuthology.orchestra.run.smithi183.stdout: "hostname": "smithi183", 2023-10-19T10:38:40.787 INFO:teuthology.orchestra.run.smithi183.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-19T10:38:40.787 INFO:teuthology.orchestra.run.smithi183.stdout: "pid": "95663", 2023-10-19T10:38:40.787 INFO:teuthology.orchestra.run.smithi183.stdout: "root": "/" 2023-10-19T10:38:40.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:38:40.788 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_count": 0, 2023-10-19T10:38:40.788 INFO:teuthology.orchestra.run.smithi183.stdout: "dentry_pinned_count": 0, 2023-10-19T10:38:40.788 INFO:teuthology.orchestra.run.smithi183.stdout: "id": 5186, 2023-10-19T10:38:40.788 INFO:teuthology.orchestra.run.smithi183.stdout: "inst": { 2023-10-19T10:38:40.789 INFO:teuthology.orchestra.run.smithi183.stdout: "name": { 2023-10-19T10:38:40.789 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "client", 2023-10-19T10:38:40.789 INFO:teuthology.orchestra.run.smithi183.stdout: "num": 5186 2023-10-19T10:38:40.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:38:40.789 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2023-10-19T10:38:40.790 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2023-10-19T10:38:40.790 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:38:40.790 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 757114590 2023-10-19T10:38:40.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-10-19T10:38:40.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:38:40.791 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": { 2023-10-19T10:38:40.791 INFO:teuthology.orchestra.run.smithi183.stdout: "type": "v1", 2023-10-19T10:38:40.791 INFO:teuthology.orchestra.run.smithi183.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:38:40.791 INFO:teuthology.orchestra.run.smithi183.stdout: "nonce": 757114590 2023-10-19T10:38:40.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-10-19T10:38:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "inst_str": "client.5186 192.168.0.1:0/757114590", 2023-10-19T10:38:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "addr_str": "192.168.0.1:0/757114590", 2023-10-19T10:38:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "inode_count": 1, 2023-10-19T10:38:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_epoch": 15, 2023-10-19T10:38:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch": 44, 2023-10-19T10:38:40.793 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:38:40.793 INFO:teuthology.orchestra.run.smithi183.stdout: "blocklisted": false, 2023-10-19T10:38:40.793 INFO:teuthology.orchestra.run.smithi183.stdout: "fs_name": "cephfs" 2023-10-19T10:38:40.793 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-10-19T10:38:40.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:40.795 DEBUG:teuthology.orchestra.run.smithi183:> stat --file-system '--printf=%T 2023-10-19T10:38:40.795 DEBUG:teuthology.orchestra.run.smithi183:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:40.810 INFO:teuthology.orchestra.run.smithi183.stdout:fuseblk 2023-10-19T10:38:40.811 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:40.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:38:40.811 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-19T10:38:40.877 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:38:40.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.993+0000 7f859dd6e640 1 Processor -- start 2023-10-19T10:38:40.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.993+0000 7f859dd6e640 1 -- start start 2023-10-19T10:38:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f859dd6e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f85980cbb20 0x7f85981609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f859dd6e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859815d450 con 0x7f8598160ee0 2023-10-19T10:38:40.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f859dd6e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859815d730 con 0x7f859815a290 2023-10-19T10:38:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f859dd6e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859806a530 con 0x7f85980cbb20 2023-10-19T10:38:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8596ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f85980cbb20 0x7f85981609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8597fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f8598160ee0 0x7f85980ccd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57624/0 (socket says 172.21.15.50:57624) 2023-10-19T10:38:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8596ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f85980cbb20 0x7f85981609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38422/0 (socket says 172.21.15.50:38422) 2023-10-19T10:38:40.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8597fff640 1 -- 172.21.15.50:0/2627968655 learned_addr learned my addr 172.21.15.50:0/2627968655 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8596ffd640 1 -- 172.21.15.50:0/2627968655 >> v1:172.21.15.183:6789/0 conn(0x7f859815a290 legacy=0x7f859815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8596ffd640 1 -- 172.21.15.50:0/2627968655 >> v1:172.21.15.50:6789/0 conn(0x7f8598160ee0 legacy=0x7f85980ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:40.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.994+0000 7f8596ffd640 1 -- 172.21.15.50:0/2627968655 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859815da10 con 0x7f85980cbb20 2023-10-19T10:38:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.995+0000 7f8596ffd640 1 --2- 172.21.15.50:0/2627968655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f85981609a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f858802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad4a9d39462bbff server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.995+0000 7f85967fc640 1 -- 172.21.15.50:0/2627968655 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588030610 con 0x7f85980cbb20 2023-10-19T10:38:41.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.995+0000 7f85967fc640 1 -- 172.21.15.50:0/2627968655 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588030770 con 0x7f85980cbb20 2023-10-19T10:38:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.995+0000 7f85967fc640 1 -- 172.21.15.50:0/2627968655 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588030b00 con 0x7f85980cbb20 2023-10-19T10:38:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 -- 172.21.15.50:0/2627968655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 msgr2=0x7f85981609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2627968655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f85981609a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f858802b960 comp rx=0 tx=0).stop 2023-10-19T10:38:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 -- 172.21.15.50:0/2627968655 shutdown_connections 2023-10-19T10:38:41.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2627968655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f85981609a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 -- 172.21.15.50:0/2627968655 >> 172.21.15.50:0/2627968655 conn(0x7f859806c9a0 msgr2=0x7f859806cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 -- 172.21.15.50:0/2627968655 shutdown_connections 2023-10-19T10:38:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.996+0000 7f859dd6e640 1 -- 172.21.15.50:0/2627968655 wait complete. 2023-10-19T10:38:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.997+0000 7f859dd6e640 1 Processor -- start 2023-10-19T10:38:41.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.997+0000 7f859dd6e640 1 -- start start 2023-10-19T10:38:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f859dd6e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f8598158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f859dd6e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f859815a290 0x7f85981f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f859dd6e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8598160ee0 0x7f85980c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f859dd6e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85980c9620 con 0x7f85980cbb20 2023-10-19T10:38:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f859dd6e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85980c6fa0 con 0x7f859815a290 2023-10-19T10:38:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f859dd6e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85980c6390 con 0x7f8598160ee0 2023-10-19T10:38:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f8598158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f8598158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38424/0 (socket says 172.21.15.50:38424) 2023-10-19T10:38:41.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 -- 172.21.15.50:0/2651885781 learned_addr learned my addr 172.21.15.50:0/2651885781 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f8596ffd640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f859815a290 0x7f85981f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f8597fff640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8598160ee0 0x7f85980c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 -- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8598160ee0 msgr2=0x7f85980c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8598160ee0 0x7f85980c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 -- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f859815a290 msgr2=0x7f85981f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f859815a290 0x7f85981f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f85977fe640 1 -- 172.21.15.50:0/2651885781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588009400 con 0x7f85980cbb20 2023-10-19T10:38:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f8596ffd640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f859815a290 0x7f85981f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.998+0000 7f8597fff640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8598160ee0 0x7f85980c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:38:41.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.999+0000 7f85977fe640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f8598158c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f858c00c450 tx=0x7f858c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.999+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c0052d0 con 0x7f85980cbb20 2023-10-19T10:38:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.999+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c005430 con 0x7f85980cbb20 2023-10-19T10:38:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.999+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85980c6670 con 0x7f85980cbb20 2023-10-19T10:38:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.999+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c012870 con 0x7f85980cbb20 2023-10-19T10:38:41.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:40.999+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85980c6c30 con 0x7f85980cbb20 2023-10-19T10:38:41.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.000+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8598073990 con 0x7f85980cbb20 2023-10-19T10:38:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.000+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f858c0129d0 con 0x7f85980cbb20 2023-10-19T10:38:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.001+0000 7f857ffff640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f858406a080 0x7f858406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.001+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f858c088190 con 0x7f85980cbb20 2023-10-19T10:38:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.001+0000 7f8596ffd640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f858406a080 0x7f858406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.002+0000 7f8596ffd640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f858406a080 0x7f858406c540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8588036000 tx=0x7f8588008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.005+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f858c053790 con 0x7f85980cbb20 2023-10-19T10:38:41.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.216+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f859806a670 con 0x7f85980cbb20 2023-10-19T10:38:41.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.216+0000 7f857ffff640 1 -- 172.21.15.50:0/2651885781 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f858c056db0 con 0x7f85980cbb20 2023-10-19T10:38:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:38:41.220 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:38:41.220 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:38:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f858406a080 msgr2=0x7f858406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f858406a080 0x7f858406c540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8588036000 tx=0x7f8588008030 comp rx=0 tx=0).stop 2023-10-19T10:38:41.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 msgr2=0x7f8598158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f8598158c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f858c00c450 tx=0x7f858c00c910 comp rx=0 tx=0).stop 2023-10-19T10:38:41.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 shutdown_connections 2023-10-19T10:38:41.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f858406a080 0x7f858406c540 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8598160ee0 0x7f85980c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f859815a290 0x7f85981f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 --2- 172.21.15.50:0/2651885781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85980cbb20 0x7f8598158c10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 >> 172.21.15.50:0/2651885781 conn(0x7f859806c9a0 msgr2=0x7f859815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 shutdown_connections 2023-10-19T10:38:41.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.218+0000 7f859dd6e640 1 -- 172.21.15.50:0/2651885781 wait complete. 2023-10-19T10:38:41.232 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_defer_session_stale false 2023-10-19T10:38:41.350 INFO:teuthology.orchestra.run.smithi050.stdout:{"success":"mds_defer_session_stale = 'false' (not observed, change may require restart) "} 2023-10-19T10:38:41.351 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_defer_session_stale = 'false' (not observed, change may require restart) " } 2023-10-19T10:38:41.351 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds defer_client_eviction_on_laggy_osds true 2023-10-19T10:38:41.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.458+0000 7f78eb4c4640 1 Processor -- start 2023-10-19T10:38:41.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.458+0000 7f78eb4c4640 1 -- start start 2023-10-19T10:38:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78eb4c4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78e40b3090 0x7f78e40d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78eb4c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e4161210 0x7f78e40b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78eb4c4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e40af250 con 0x7f78e40b38f0 2023-10-19T10:38:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78eb4c4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e40b0190 con 0x7f78e40b3090 2023-10-19T10:38:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78eb4c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40af510 con 0x7f78e4161210 2023-10-19T10:38:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78e8a5f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78e40b3090 0x7f78e40d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78e8a5f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78e40b3090 0x7f78e40d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38426/0 (socket says 172.21.15.50:38426) 2023-10-19T10:38:41.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.459+0000 7f78e8a5f640 1 -- 172.21.15.50:0/3718645671 learned_addr learned my addr 172.21.15.50:0/3718645671 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:41.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e9a61640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e4161210 0x7f78e40b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e8a5f640 1 -- 172.21.15.50:0/3718645671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e4161210 msgr2=0x7f78e40b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:41.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e8a5f640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e4161210 0x7f78e40b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e8a5f640 1 -- 172.21.15.50:0/3718645671 >> v1:172.21.15.183:6789/0 conn(0x7f78e40b38f0 legacy=0x7f78e4160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:41.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e8a5f640 1 -- 172.21.15.50:0/3718645671 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e40af7f0 con 0x7f78e40b3090 2023-10-19T10:38:41.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e9a61640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e4161210 0x7f78e40b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:38:41.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.460+0000 7f78e8a5f640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e40b3090 0x7f78e40d30e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009870 tx=0x7f78d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cb45a2db6752cf server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78e3fff640 1 -- 172.21.15.50:0/3718645671 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8031640 con 0x7f78e40b3090 2023-10-19T10:38:41.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78e3fff640 1 -- 172.21.15.50:0/3718645671 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d80317a0 con 0x7f78e40b3090 2023-10-19T10:38:41.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78e3fff640 1 -- 172.21.15.50:0/3718645671 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8031b30 con 0x7f78e40b3090 2023-10-19T10:38:41.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 -- 172.21.15.50:0/3718645671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e40b3090 msgr2=0x7f78e40d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e40b3090 0x7f78e40d30e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009870 tx=0x7f78d802b8f0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 -- 172.21.15.50:0/3718645671 shutdown_connections 2023-10-19T10:38:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e4161210 0x7f78e40b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/3718645671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e40b3090 0x7f78e40d30e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 -- 172.21.15.50:0/3718645671 >> 172.21.15.50:0/3718645671 conn(0x7f78e409b960 msgr2=0x7f78e40d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:41.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 -- 172.21.15.50:0/3718645671 shutdown_connections 2023-10-19T10:38:41.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.461+0000 7f78eb4c4640 1 -- 172.21.15.50:0/3718645671 wait complete. 2023-10-19T10:38:41.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.462+0000 7f78eb4c4640 1 Processor -- start 2023-10-19T10:38:41.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.462+0000 7f78eb4c4640 1 -- start start 2023-10-19T10:38:41.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78eb4c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78e40b3090 0x7f78e40cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78eb4c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e40b38f0 0x7f78e4154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78eb4c4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 0x7f78e41f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78eb4c4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4155d30 con 0x7f78e4161210 2023-10-19T10:38:41.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78eb4c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40cc290 con 0x7f78e40b3090 2023-10-19T10:38:41.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78eb4c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40cc570 con 0x7f78e40b38f0 2023-10-19T10:38:41.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78e9a61640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 0x7f78e41f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78e9a61640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 0x7f78e41f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38430/0 (socket says 172.21.15.50:38430) 2023-10-19T10:38:41.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78e9a61640 1 -- 172.21.15.50:0/66659390 learned_addr learned my addr 172.21.15.50:0/66659390 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:41.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78e8a5f640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e40b38f0 0x7f78e4154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.463+0000 7f78e9260640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78e40b3090 0x7f78e40cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e9a61640 1 -- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e40b38f0 msgr2=0x7f78e4154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e9a61640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e40b38f0 0x7f78e4154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e9a61640 1 -- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78e40b3090 msgr2=0x7f78e40cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e9a61640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78e40b3090 0x7f78e40cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e9a61640 1 -- 172.21.15.50:0/66659390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80094d0 con 0x7f78e4161210 2023-10-19T10:38:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e9a61640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 0x7f78e41f5460 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78d400ccf0 tx=0x7f78d400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d400f680 con 0x7f78e4161210 2023-10-19T10:38:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d400f800 con 0x7f78e4161210 2023-10-19T10:38:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d400fbb0 con 0x7f78e4161210 2023-10-19T10:38:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e40cc850 con 0x7f78e4161210 2023-10-19T10:38:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.464+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e41559c0 con 0x7f78e4161210 2023-10-19T10:38:41.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.465+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e4155ea0 con 0x7f78e4161210 2023-10-19T10:38:41.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.465+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f78d400b040 con 0x7f78e4161210 2023-10-19T10:38:41.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.466+0000 7f78e1ffb640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f78d006a080 0x7f78d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.466+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f78d4089090 con 0x7f78e4161210 2023-10-19T10:38:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.466+0000 7f78e9260640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f78d006a080 0x7f78d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.467+0000 7f78e9260640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f78d006a080 0x7f78d006c540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f78dc004e50 tx=0x7f78dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.470+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f78d4054600 con 0x7f78e4161210 2023-10-19T10:38:41.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.646+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=defer_client_eviction_on_laggy_osds}] v 0) v1 -- 0x7f78e404e420 con 0x7f78e4161210 2023-10-19T10:38:41.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.649+0000 7f78e1ffb640 1 -- 172.21.15.50:0/66659390 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=defer_client_eviction_on_laggy_osds}]=0 v11)=0 v11) v1 ==== 136+0+0 (secure 0 0 0) 0x7f78d4057c20 con 0x7f78e4161210 2023-10-19T10:38:41.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f78d006a080 msgr2=0x7f78d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f78d006a080 0x7f78d006c540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f78dc004e50 tx=0x7f78dc009040 comp rx=0 tx=0).stop 2023-10-19T10:38:41.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 msgr2=0x7f78e41f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 0x7f78e41f5460 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78d400ccf0 tx=0x7f78d400a650 comp rx=0 tx=0).stop 2023-10-19T10:38:41.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 shutdown_connections 2023-10-19T10:38:41.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f78d006a080 0x7f78d006c540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e4161210 0x7f78e41f5460 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78e40b38f0 0x7f78e4154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 --2- 172.21.15.50:0/66659390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78e40b3090 0x7f78e40cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 >> 172.21.15.50:0/66659390 conn(0x7f78e409b960 msgr2=0x7f78e40cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:41.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 shutdown_connections 2023-10-19T10:38:41.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.651+0000 7f78eb4c4640 1 -- 172.21.15.50:0/66659390 wait complete. 2023-10-19T10:38:41.664 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds defer_client_eviction_on_laggy_osds 2023-10-19T10:38:41.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.770+0000 7fc07fa48640 1 Processor -- start 2023-10-19T10:38:41.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.770+0000 7fc07fa48640 1 -- start start 2023-10-19T10:38:41.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07fa48640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc07815c620 0x7fc0780b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07fa48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 0x7fc0780cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07fa48640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0780cda30 con 0x7fc0780b55c0 2023-10-19T10:38:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07fa48640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0780ccc10 con 0x7fc07815c620 2023-10-19T10:38:41.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07fa48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc078161110 con 0x7fc0780b82f0 2023-10-19T10:38:41.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07dfe5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc0780b55c0 0x7fc0780b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57638/0 (socket says 172.21.15.50:57638) 2023-10-19T10:38:41.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07dfe5640 1 -- 172.21.15.50:0/377028363 learned_addr learned my addr 172.21.15.50:0/377028363 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07cfe3640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 0x7fc0780cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc07d7e4640 1 --2- 172.21.15.50:0/377028363 >> v2:172.21.15.183:3300/0 conn(0x7fc07815c620 0x7fc0780b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971824265 0 0) 0x7fc0780cda30 con 0x7fc0780b55c0 2023-10-19T10:38:41.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc064003520 con 0x7fc0780b55c0 2023-10-19T10:38:41.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 188179895 0 0) 0x7fc064003520 con 0x7fc0780b55c0 2023-10-19T10:38:41.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 msgr2=0x7fc0780cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:41.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 0x7fc0780cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 >> v2:172.21.15.183:3300/0 conn(0x7fc07815c620 msgr2=0x7fc0780b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:41.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 --2- 172.21.15.50:0/377028363 >> v2:172.21.15.183:3300/0 conn(0x7fc07815c620 0x7fc0780b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780cce20 con 0x7fc0780b55c0 2023-10-19T10:38:41.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fc0700019f0 con 0x7fc0780b55c0 2023-10-19T10:38:41.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 >> v1:172.21.15.50:6789/0 conn(0x7fc0780b55c0 legacy=0x7fc0780b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.771+0000 7fc06ffff640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 0x7fc064003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc06ffff640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064003ef0 0x7fc0640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc06ffff640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc064005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780ccc10 con 0x7fc064004d00 2023-10-19T10:38:41.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078161110 con 0x7fc064003ef0 2023-10-19T10:38:41.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06400b980 con 0x7fc0780b82f0 2023-10-19T10:38:41.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07dfe5640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc064005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07cfe3640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064003ef0 0x7fc0640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07d7e4640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 0x7fc064003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07dfe5640 1 -- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 msgr2=0x7fc064003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07dfe5640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0780b82f0 0x7fc064003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07dfe5640 1 -- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064003ef0 msgr2=0x7fc0640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07dfe5640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064003ef0 0x7fc0640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.772+0000 7fc07dfe5640 1 -- 172.21.15.50:0/377028363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc064003520 con 0x7fc064004d00 2023-10-19T10:38:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc07dfe5640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc064005180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc0780b5a10 tx=0x7fc0640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbc500a6164e0a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070034630 con 0x7fc064004d00 2023-10-19T10:38:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07002f070 con 0x7fc064004d00 2023-10-19T10:38:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc06ffff640 1 -- 172.21.15.50:0/377028363 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700349d0 con 0x7fc064004d00 2023-10-19T10:38:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc07fa48640 1 -- 172.21.15.50:0/377028363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 msgr2=0x7fc064005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc07fa48640 1 --2- 172.21.15.50:0/377028363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc064005180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc0780b5a10 tx=0x7fc0640057f0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc07dfe5640 1 -- 172.21.15.50:0/377028363 reap_dead start 2023-10-19T10:38:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc07fa48640 1 -- 172.21.15.50:0/377028363 shutdown_connections 2023-10-19T10:38:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.773+0000 7fc07fa48640 1 -- 172.21.15.50:0/377028363 >> 172.21.15.50:0/377028363 conn(0x7fc078071110 msgr2=0x7fc0780714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.774+0000 7fc07fa48640 1 -- 172.21.15.50:0/377028363 shutdown_connections 2023-10-19T10:38:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.774+0000 7fc07fa48640 1 -- 172.21.15.50:0/377028363 wait complete. 2023-10-19T10:38:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 Processor -- start 2023-10-19T10:38:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 -- start start 2023-10-19T10:38:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc0780c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc078152da0 0x7fc078153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0780cba10 0x7fc0780cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780cc3d0 con 0x7fc064004d00 2023-10-19T10:38:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078153910 con 0x7fc0780cba10 2023-10-19T10:38:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.775+0000 7fc07fa48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc078153bf0 con 0x7fc078152da0 2023-10-19T10:38:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc0780c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc0780c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38452/0 (socket says 172.21.15.50:38452) 2023-10-19T10:38:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 -- 172.21.15.50:0/4082007895 learned_addr learned my addr 172.21.15.50:0/4082007895 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07cfe3640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc078152da0 0x7fc078153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07dfe5640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0780cba10 0x7fc0780cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 -- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc078152da0 msgr2=0x7fc078153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc078152da0 0x7fc078153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 -- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0780cba10 msgr2=0x7fc0780cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0780cba10 0x7fc0780cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 -- 172.21.15.50:0/4082007895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078151590 con 0x7fc064004d00 2023-10-19T10:38:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07d7e4640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc0780c9690 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc074011a70 tx=0x7fc074011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074009680 con 0x7fc064004d00 2023-10-19T10:38:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07400fa90 con 0x7fc064004d00 2023-10-19T10:38:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc078151870 con 0x7fc064004d00 2023-10-19T10:38:41.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07400fe20 con 0x7fc064004d00 2023-10-19T10:38:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.776+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0781f57f0 con 0x7fc064004d00 2023-10-19T10:38:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.777+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc078153e00 con 0x7fc064004d00 2023-10-19T10:38:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.778+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc074014440 con 0x7fc064004d00 2023-10-19T10:38:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.778+0000 7fc06dffb640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc05c06a050 0x7fc05c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.778+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc074088f10 con 0x7fc064004d00 2023-10-19T10:38:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.779+0000 7fc07cfe3640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc05c06a050 0x7fc05c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.779+0000 7fc07cfe3640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc05c06a050 0x7fc05c06c510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0780b7400 tx=0x7fc068009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.783+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc074054480 con 0x7fc064004d00 2023-10-19T10:38:41.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.961+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "defer_client_eviction_on_laggy_osds"} v 0) v1 -- 0x7fc078073990 con 0x7fc064004d00 2023-10-19T10:38:41.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.962+0000 7fc06dffb640 1 -- 172.21.15.50:0/4082007895 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "defer_client_eviction_on_laggy_osds"}]=0 v11) v1 ==== 118+0+5 (secure 0 0 0) 0x7fc074057aa0 con 0x7fc064004d00 2023-10-19T10:38:41.965 INFO:teuthology.orchestra.run.smithi050.stdout:true 2023-10-19T10:38:41.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc05c06a050 msgr2=0x7fc05c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc05c06a050 0x7fc05c06c510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0780b7400 tx=0x7fc068009190 comp rx=0 tx=0).stop 2023-10-19T10:38:41.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 msgr2=0x7fc0780c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:41.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc0780c9690 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc074011a70 tx=0x7fc074011e70 comp rx=0 tx=0).stop 2023-10-19T10:38:41.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 shutdown_connections 2023-10-19T10:38:41.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc05c06a050 0x7fc05c06c510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0780cba10 0x7fc0780cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc078152da0 0x7fc078153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 --2- 172.21.15.50:0/4082007895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc064004d00 0x7fc0780c9690 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:41.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 >> 172.21.15.50:0/4082007895 conn(0x7fc078071110 msgr2=0x7fc0780d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:41.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 shutdown_connections 2023-10-19T10:38:41.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:41.964+0000 7fc07fa48640 1 -- 172.21.15.50:0/4082007895 wait complete. 2023-10-19T10:38:41.977 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-10-19T10:38:42.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.081+0000 7f9bdfe46640 1 Processor -- start 2023-10-19T10:38:42.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.081+0000 7f9bdfe46640 1 -- start start 2023-10-19T10:38:42.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdfe46640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9bd815c570 0x7f9bd80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:42.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdfe46640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd80cd920 0x7f9bd80d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:42.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdfe46640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd806aa30 con 0x7f9bd815c950 2023-10-19T10:38:42.085 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdfe46640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd806b2d0 con 0x7f9bd815c570 2023-10-19T10:38:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdfe46640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80ce090 con 0x7f9bd80cd920 2023-10-19T10:38:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bddbe2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9bd815c570 0x7f9bd80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bde3e3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd80cd920 0x7f9bd80d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdd3e1640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f9bd815c950 0x7f9bd806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:59682/0 (socket says 172.21.15.50:59682) 2023-10-19T10:38:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.082+0000 7f9bdd3e1640 1 -- 172.21.15.50:0/2412323629 learned_addr learned my addr 172.21.15.50:0/2412323629 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:38:42.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.083+0000 7f9bde3e3640 1 -- 172.21.15.50:0/2412323629 >> v2:172.21.15.183:3300/0 conn(0x7f9bd815c570 msgr2=0x7f9bd80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:38:42.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.083+0000 7f9bde3e3640 1 --2- 172.21.15.50:0/2412323629 >> v2:172.21.15.183:3300/0 conn(0x7f9bd815c570 0x7f9bd80d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.088 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.083+0000 7f9bde3e3640 1 -- 172.21.15.50:0/2412323629 >> v1:172.21.15.183:6789/0 conn(0x7f9bd815c950 legacy=0x7f9bd806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.083+0000 7f9bde3e3640 1 -- 172.21.15.50:0/2412323629 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd806ac80 con 0x7f9bd80cd920 2023-10-19T10:38:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.083+0000 7f9bddbe2640 1 --2- 172.21.15.50:0/2412323629 >> v2:172.21.15.183:3300/0 conn(0x7f9bd815c570 0x7f9bd80d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:38:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bde3e3640 1 --2- 172.21.15.50:0/2412323629 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd80cd920 0x7f9bd80d26a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009d90 tx=0x7f9bd002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5fcb8078d55c77 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdcbe0640 1 -- 172.21.15.50:0/2412323629 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd002cbc0 con 0x7f9bd80cd920 2023-10-19T10:38:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdcbe0640 1 -- 172.21.15.50:0/2412323629 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd002cd20 con 0x7f9bd80cd920 2023-10-19T10:38:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdcbe0640 1 -- 172.21.15.50:0/2412323629 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0034660 con 0x7f9bd80cd920 2023-10-19T10:38:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdfe46640 1 -- 172.21.15.50:0/2412323629 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd80cd920 msgr2=0x7f9bd80d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/2412323629 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd80cd920 0x7f9bd80d26a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009d90 tx=0x7f9bd002be10 comp rx=0 tx=0).stop 2023-10-19T10:38:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdfe46640 1 -- 172.21.15.50:0/2412323629 shutdown_connections 2023-10-19T10:38:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/2412323629 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd80cd920 0x7f9bd80d26a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/2412323629 >> v2:172.21.15.183:3300/0 conn(0x7f9bd815c570 0x7f9bd80d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.084+0000 7f9bdfe46640 1 -- 172.21.15.50:0/2412323629 >> 172.21.15.50:0/2412323629 conn(0x7f9bd8089ff0 msgr2=0x7f9bd809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.085+0000 7f9bdfe46640 1 -- 172.21.15.50:0/2412323629 shutdown_connections 2023-10-19T10:38:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.085+0000 7f9bdfe46640 1 -- 172.21.15.50:0/2412323629 wait complete. 2023-10-19T10:38:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.086+0000 7f9bdfe46640 1 Processor -- start 2023-10-19T10:38:42.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.086+0000 7f9bdfe46640 1 -- start start 2023-10-19T10:38:42.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.086+0000 7f9bdfe46640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 0x7f9bd8149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:42.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.086+0000 7f9bdfe46640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd815c570 0x7f9bd80ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:42.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdfe46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd815c950 0x7f9bd80cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:42.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdfe46640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80c9980 con 0x7f9bd80cd920 2023-10-19T10:38:42.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdfe46640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80c9c60 con 0x7f9bd815c950 2023-10-19T10:38:42.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdfe46640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80c9f40 con 0x7f9bd815c570 2023-10-19T10:38:42.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 0x7f9bd8149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:42.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 0x7f9bd8149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38462/0 (socket says 172.21.15.50:38462) 2023-10-19T10:38:42.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 -- 172.21.15.50:0/3570021570 learned_addr learned my addr 172.21.15.50:0/3570021570 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:38:42.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdd3e1640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd815c570 0x7f9bd80ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:42.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bde3e3640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd815c950 0x7f9bd80cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:42.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 -- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd815c570 msgr2=0x7f9bd80ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:42.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd815c570 0x7f9bd80ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 -- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd815c950 msgr2=0x7f9bd80cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:42.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd815c950 0x7f9bd80cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 -- 172.21.15.50:0/3570021570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd40093a0 con 0x7f9bd80cd920 2023-10-19T10:38:42.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bddbe2640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 0x7f9bd8149ca0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400ebe0 tx=0x7f9bd400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:42.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4009750 con 0x7f9bd80cd920 2023-10-19T10:38:42.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd400cc30 con 0x7f9bd80cd920 2023-10-19T10:38:42.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd0009a70 con 0x7f9bd80cd920 2023-10-19T10:38:42.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400eee0 con 0x7f9bd80cd920 2023-10-19T10:38:42.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.087+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80ca500 con 0x7f9bd80cd920 2023-10-19T10:38:42.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.089+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd80ca6c0 con 0x7f9bd80cd920 2023-10-19T10:38:42.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.089+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9bd4005990 con 0x7f9bd80cd920 2023-10-19T10:38:42.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.089+0000 7f9bce7fc640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9bbc06a080 0x7f9bbc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:38:42.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.089+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9bd4088b80 con 0x7f9bd80cd920 2023-10-19T10:38:42.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.090+0000 7f9bdd3e1640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9bbc06a080 0x7f9bbc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:38:42.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.090+0000 7f9bdd3e1640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9bbc06a080 0x7f9bbc06c540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9bd81f4eb0 tx=0x7f9bc8007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:38:42.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.094+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9bd40541a0 con 0x7f9bd80cd920 2023-10-19T10:38:42.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.271+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f9bd8073990 con 0x7f9bd80cd920 2023-10-19T10:38:42.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.271+0000 7f9bce7fc640 1 -- 172.21.15.50:0/3570021570 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v44) v1 ==== 56+0+3489 (secure 0 0 0) 0x7f9bd4020dc0 con 0x7f9bd80cd920 2023-10-19T10:38:42.274 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 44 2023-10-19T10:38:42.275 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:38:42.275 INFO:teuthology.orchestra.run.smithi050.stdout:created 2023-10-19T10:36:30.337960+0000 2023-10-19T10:38:42.275 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2023-10-19T10:38:37.117837+0000 2023-10-19T10:38:42.275 INFO:teuthology.orchestra.run.smithi050.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-19T10:38:42.275 INFO:teuthology.orchestra.run.smithi050.stdout:crush_version 5 2023-10-19T10:38:42.276 INFO:teuthology.orchestra.run.smithi050.stdout:full_ratio 0.95 2023-10-19T10:38:42.276 INFO:teuthology.orchestra.run.smithi050.stdout:backfillfull_ratio 0.9 2023-10-19T10:38:42.276 INFO:teuthology.orchestra.run.smithi050.stdout:nearfull_ratio 0.85 2023-10-19T10:38:42.276 INFO:teuthology.orchestra.run.smithi050.stdout:require_min_compat_client luminous 2023-10-19T10:38:42.276 INFO:teuthology.orchestra.run.smithi050.stdout:min_compat_client luminous 2023-10-19T10:38:42.276 INFO:teuthology.orchestra.run.smithi050.stdout:require_osd_release reef 2023-10-19T10:38:42.277 INFO:teuthology.orchestra.run.smithi050.stdout:stretch_mode_enabled false 2023-10-19T10:38:42.277 INFO:teuthology.orchestra.run.smithi050.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2023-10-19T10:38:42.277 INFO:teuthology.orchestra.run.smithi050.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 application rbd read_balance_score 1.99 2023-10-19T10:38:42.277 INFO:teuthology.orchestra.run.smithi050.stdout:pool 6 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 41 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.75 2023-10-19T10:38:42.277 INFO:teuthology.orchestra.run.smithi050.stdout:pool 7 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 42 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.63 2023-10-19T10:38:42.278 INFO:teuthology.orchestra.run.smithi050.stdout:max_osd 8 2023-10-19T10:38:42.278 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] [v2:172.21.15.50:6817/231212571,v1:172.21.15.50:6819/231212571] exists,up 969faf31-afdd-4ca1-862d-266c0a9d117f 2023-10-19T10:38:42.278 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] [v2:172.21.15.50:6826/3823862985,v1:172.21.15.50:6827/3823862985] exists,up 446ff7aa-e634-46bf-aba0-fa0084bc4d57 2023-10-19T10:38:42.278 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] [v2:172.21.15.50:6808/415608244,v1:172.21.15.50:6809/415608244] exists,up 6852e173-ea11-407a-bc20-c9f6669740c3 2023-10-19T10:38:42.278 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] [v2:172.21.15.50:6802/945885036,v1:172.21.15.50:6803/945885036] exists,up b4f1c130-4798-40fa-a122-8b38a2ea0d8b 2023-10-19T10:38:42.279 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] [v2:172.21.15.183:6802/2392619067,v1:172.21.15.183:6803/2392619067] exists,up dbc106ec-a15b-4b9c-82f1-b6e1cefceef0 2023-10-19T10:38:42.279 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 up in weight 1 up_from 13 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] [v2:172.21.15.183:6810/3486412186,v1:172.21.15.183:6811/3486412186] exists,up de88c853-3324-48a2-a105-bf15a3dea21c 2023-10-19T10:38:42.279 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 up in weight 1 up_from 13 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] [v2:172.21.15.183:6818/265753870,v1:172.21.15.183:6819/265753870] exists,up 2acb3a97-1331-427e-a927-b37554c81873 2023-10-19T10:38:42.279 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 up in weight 1 up_from 14 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] [v2:172.21.15.183:6826/3403463202,v1:172.21.15.183:6827/3403463202] exists,up 956a9722-f431-4874-b98b-c3be06fb3e14 2023-10-19T10:38:42.279 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 7.6 [0,5] 2023-10-19T10:38:42.280 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 7.30 [0,4] 2023-10-19T10:38:42.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.273+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9bbc06a080 msgr2=0x7f9bbc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.273+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9bbc06a080 0x7f9bbc06c540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9bd81f4eb0 tx=0x7f9bc8007e80 comp rx=0 tx=0).stop 2023-10-19T10:38:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.273+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 msgr2=0x7f9bd8149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:38:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.273+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 0x7f9bd8149ca0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400ebe0 tx=0x7f9bd400c520 comp rx=0 tx=0).stop 2023-10-19T10:38:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 shutdown_connections 2023-10-19T10:38:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9bbc06a080 0x7f9bbc06c540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd815c950 0x7f9bd80cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bd815c570 0x7f9bd80ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 --2- 172.21.15.50:0/3570021570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd80cd920 0x7f9bd8149ca0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:38:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 >> 172.21.15.50:0/3570021570 conn(0x7f9bd8089ff0 msgr2=0x7f9bd815ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:38:42.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 shutdown_connections 2023-10-19T10:38:42.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:38:42.274+0000 7f9bdfe46640 1 -- 172.21.15.50:0/3570021570 wait complete. 2023-10-19T10:38:42.286 INFO:tasks.cephfs.filesystem.ceph_manager:['osd.0 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] [v2:172.21.15.50:6817/231212571,v1:172.21.15.50:6819/231212571] exists,up 969faf31-afdd-4ca1-862d-266c0a9d117f', 'osd.1 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] [v2:172.21.15.50:6826/3823862985,v1:172.21.15.50:6827/3823862985] exists,up 446ff7aa-e634-46bf-aba0-fa0084bc4d57', 'osd.2 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6804/415608244,v1:172.21.15.50:6806/415608244] [v2:172.21.15.50:6808/415608244,v1:172.21.15.50:6809/415608244] exists,up 6852e173-ea11-407a-bc20-c9f6669740c3', 'osd.3 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] [v2:172.21.15.50:6802/945885036,v1:172.21.15.50:6803/945885036] exists,up b4f1c130-4798-40fa-a122-8b38a2ea0d8b', 'osd.4 up in weight 1 up_from 13 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] [v2:172.21.15.183:6802/2392619067,v1:172.21.15.183:6803/2392619067] exists,up dbc106ec-a15b-4b9c-82f1-b6e1cefceef0', 'osd.5 up in weight 1 up_from 13 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] [v2:172.21.15.183:6810/3486412186,v1:172.21.15.183:6811/3486412186] exists,up de88c853-3324-48a2-a105-bf15a3dea21c', 'osd.6 up in weight 1 up_from 13 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] [v2:172.21.15.183:6818/265753870,v1:172.21.15.183:6819/265753870] exists,up 2acb3a97-1331-427e-a927-b37554c81873', 'osd.7 up in weight 1 up_from 14 up_thru 40 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] [v2:172.21.15.183:6826/3403463202,v1:172.21.15.183:6827/3403463202] exists,up 956a9722-f431-4874-b98b-c3be06fb3e14'] 2023-10-19T10:38:42.287 INFO:tasks.ceph.osd.2:Sent signal 19 2023-10-19T10:38:55.969 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:38:55.978 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:39:03.783 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:39:03.781+0000 7f5c708b1640 -1 osd.6 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.647904+0000 front 2023-10-19T10:38:38.647918+0000 (oldest deadline 2023-10-19T10:39:03.347693+0000) 2023-10-19T10:39:04.205 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:39:04.203+0000 7f1b3ea9f640 -1 osd.1 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.005387+0000 front 2023-10-19T10:38:41.005318+0000 (oldest deadline 2023-10-19T10:39:03.905407+0000) 2023-10-19T10:39:04.392 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:39:04.389+0000 7fbcd3194640 -1 osd.7 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.455734+0000 front 2023-10-19T10:38:38.455801+0000 (oldest deadline 2023-10-19T10:39:04.355722+0000) 2023-10-19T10:39:04.578 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:39:04.576+0000 7fd5b6627640 -1 osd.4 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.351843+0000 front 2023-10-19T10:38:38.351664+0000 (oldest deadline 2023-10-19T10:39:04.251553+0000) 2023-10-19T10:39:04.808 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:39:04.806+0000 7f5c708b1640 -1 osd.6 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.647904+0000 front 2023-10-19T10:38:38.647918+0000 (oldest deadline 2023-10-19T10:39:03.347693+0000) 2023-10-19T10:39:05.218 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:39:05.215+0000 7f1b3ea9f640 -1 osd.1 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.005387+0000 front 2023-10-19T10:38:41.005318+0000 (oldest deadline 2023-10-19T10:39:03.905407+0000) 2023-10-19T10:39:05.371 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:39:05.369+0000 7fbcd3194640 -1 osd.7 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.455734+0000 front 2023-10-19T10:38:38.455801+0000 (oldest deadline 2023-10-19T10:39:04.355722+0000) 2023-10-19T10:39:05.562 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:39:05.560+0000 7fd5b6627640 -1 osd.4 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.351843+0000 front 2023-10-19T10:38:38.351664+0000 (oldest deadline 2023-10-19T10:39:04.251553+0000) 2023-10-19T10:39:05.842 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:39:05.840+0000 7f5c708b1640 -1 osd.6 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.647904+0000 front 2023-10-19T10:38:38.647918+0000 (oldest deadline 2023-10-19T10:39:03.347693+0000) 2023-10-19T10:39:06.169 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:39:06.167+0000 7f1b3ea9f640 -1 osd.1 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.005387+0000 front 2023-10-19T10:38:41.005318+0000 (oldest deadline 2023-10-19T10:39:03.905407+0000) 2023-10-19T10:39:06.388 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:39:06.385+0000 7fbcd3194640 -1 osd.7 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.455734+0000 front 2023-10-19T10:38:38.455801+0000 (oldest deadline 2023-10-19T10:39:04.355722+0000) 2023-10-19T10:39:06.610 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:39:06.608+0000 7fd5b6627640 -1 osd.4 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.351843+0000 front 2023-10-19T10:38:38.351664+0000 (oldest deadline 2023-10-19T10:39:04.251553+0000) 2023-10-19T10:39:06.838 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-10-19T10:39:06.835+0000 7f5c708b1640 -1 osd.6 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.647904+0000 front 2023-10-19T10:38:38.647918+0000 (oldest deadline 2023-10-19T10:39:03.347693+0000) 2023-10-19T10:39:06.906 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-10-19T10:39:06.903+0000 7efc0867f640 -1 osd.5 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.134603+0000 front 2023-10-19T10:38:41.134556+0000 (oldest deadline 2023-10-19T10:39:06.434479+0000) 2023-10-19T10:39:07.141 INFO:tasks.ceph.osd.0.smithi050.stderr:2023-10-19T10:39:07.138+0000 7efc3465f640 -1 osd.0 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.438040+0000 front 2023-10-19T10:38:41.438020+0000 (oldest deadline 2023-10-19T10:39:06.738065+0000) 2023-10-19T10:39:07.152 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:39:07.150+0000 7f1b3ea9f640 -1 osd.1 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.005387+0000 front 2023-10-19T10:38:41.005318+0000 (oldest deadline 2023-10-19T10:39:03.905407+0000) 2023-10-19T10:39:07.356 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-10-19T10:39:07.354+0000 7fbcd3194640 -1 osd.7 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.455734+0000 front 2023-10-19T10:38:38.455801+0000 (oldest deadline 2023-10-19T10:39:04.355722+0000) 2023-10-19T10:39:07.615 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-10-19T10:39:07.613+0000 7fd5b6627640 -1 osd.4 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:38.351843+0000 front 2023-10-19T10:38:38.351664+0000 (oldest deadline 2023-10-19T10:39:04.251553+0000) 2023-10-19T10:39:07.924 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-10-19T10:39:07.922+0000 7efc0867f640 -1 osd.5 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.134603+0000 front 2023-10-19T10:38:41.134556+0000 (oldest deadline 2023-10-19T10:39:06.434479+0000) 2023-10-19T10:39:08.103 INFO:tasks.ceph.osd.1.smithi050.stderr:2023-10-19T10:39:08.100+0000 7f1b3ea9f640 -1 osd.1 44 heartbeat_check: no reply from 172.21.15.50:6811 osd.2 since back 2023-10-19T10:38:41.005387+0000 front 2023-10-19T10:38:41.005318+0000 (oldest deadline 2023-10-19T10:39:03.905407+0000) 2023-10-19T10:39:26.038 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:39:26.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:39:56.112 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:39:56.116 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:40:26.192 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:40:26.196 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:40:42.289 INFO:tasks.ceph.osd.2:Sent signal 18 2023-10-19T10:40:42.289 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:40:42.290 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-19T10:40:42.290 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-19T10:40:42.291 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-19T10:40:42.291 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-19T10:40:42.291 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:40:42.292 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-19T10:40:42.292 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-19T10:40:42.292 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-19T10:40:42.293 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-19T10:40:42.293 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:40:42.293 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-19T10:40:42.294 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-19T10:40:42.294 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-19T10:40:42.294 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-19T10:40:42.295 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:40:42.295 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-19T10:40:42.296 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:40:42.296 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:40:42.296 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:40:42.297 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-19T10:40:42.297 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-19T10:40:42.297 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-19T10:40:42.298 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:40:42.298 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:40:42.298 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.0")) 2023-10-19T10:40:42.299 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-19T10:40:42.398 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.0.105851.asok 2023-10-19T10:40:42.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:40:42.409 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105851.asok 2023-10-19T10:40:42.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:40:42.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105851.asok mds_sessions 2023-10-19T10:40:42.552 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:40:42.552 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5994, 2023-10-19T10:40:42.552 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-19T10:40:42.553 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-19T10:40:42.553 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-19T10:40:42.553 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 5994 2023-10-19T10:40:42.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:40:42.553 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:40:42.554 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:40:42.554 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:40:42.554 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 216420809 2023-10-19T10:40:42.554 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:40:42.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:40:42.555 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.5994 v1:192.168.0.1:0/216420809", 2023-10-19T10:40:42.555 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "v1:192.168.0.1:0/216420809", 2023-10-19T10:40:42.555 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions": [ 2023-10-19T10:40:42.555 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:40:42.555 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": 0, 2023-10-19T10:40:42.555 INFO:teuthology.orchestra.run.smithi050.stdout: "addrs": { 2023-10-19T10:40:42.556 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2023-10-19T10:40:42.556 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:40:42.556 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2023-10-19T10:40:42.556 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6832", 2023-10-19T10:40:42.556 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1814941523 2023-10-19T10:40:42.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:40:42.557 INFO:teuthology.orchestra.run.smithi050.stdout: { 2023-10-19T10:40:42.557 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:40:42.557 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6834", 2023-10-19T10:40:42.557 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1814941523 2023-10-19T10:40:42.557 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:40:42.558 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2023-10-19T10:40:42.558 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:40:42.558 INFO:teuthology.orchestra.run.smithi050.stdout: "seq": 0, 2023-10-19T10:40:42.558 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_gen": 0, 2023-10-19T10:40:42.558 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_ttl": "2023-10-19T10:41:39.891451+0000", 2023-10-19T10:40:42.559 INFO:teuthology.orchestra.run.smithi050.stdout: "last_cap_renew_request": "2023-10-19T10:40:39.891451+0000", 2023-10-19T10:40:42.559 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_renew_seq": 8, 2023-10-19T10:40:42.559 INFO:teuthology.orchestra.run.smithi050.stdout: "num_caps": 1, 2023-10-19T10:40:42.559 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "open" 2023-10-19T10:40:42.559 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:40:42.560 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2023-10-19T10:40:42.560 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 15 2023-10-19T10:40:42.560 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:40:42.561 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-19T10:40:42.561 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:40:42.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:40:42.562 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 down 2023-10-19T10:40:42.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:40:42.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.736+0000 7f0fb2641640 1 Processor -- start 2023-10-19T10:40:42.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.736+0000 7f0fb2641640 1 -- start start 2023-10-19T10:40:42.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb2641640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0fac0b1f30 0x7f0fac0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb2641640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fac160340 0x7f0fac1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb2641640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac15ee00 con 0x7f0fac06f270 2023-10-19T10:40:42.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb2641640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac15d6e0 con 0x7f0fac0b1f30 2023-10-19T10:40:42.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb2641640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0b2850 con 0x7f0fac160340 2023-10-19T10:40:42.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb0bde640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f0fac06f270 0x7f0fac06f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48642/0 (socket says 172.21.15.50:48642) 2023-10-19T10:40:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1081180131 learned_addr learned my addr 172.21.15.50:0/1081180131 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:40:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454200588 0 0) 0x7f0fac15ee00 con 0x7f0fac06f270 2023-10-19T10:40:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f94003520 con 0x7f0fac06f270 2023-10-19T10:40:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fab7fe640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fac160340 0x7f0fac1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.737+0000 7f0fabfff640 1 --2- 172.21.15.50:0/1081180131 >> v2:172.21.15.183:3300/0 conn(0x7f0fac0b1f30 0x7f0fac0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606414333 0 0) 0x7f0f94003520 con 0x7f0fac06f270 2023-10-19T10:40:42.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fac160340 msgr2=0x7f0fac1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:40:42.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fac160340 0x7f0fac1607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 >> v2:172.21.15.183:3300/0 conn(0x7f0fac0b1f30 msgr2=0x7f0fac0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:40:42.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 --2- 172.21.15.50:0/1081180131 >> v2:172.21.15.183:3300/0 conn(0x7f0fac0b1f30 0x7f0fac0b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.745 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac15f030 con 0x7f0fac06f270 2023-10-19T10:40:42.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f0fa40019f0 con 0x7f0fac06f270 2023-10-19T10:40:42.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fabfff640 1 --2- 172.21.15.50:0/1081180131 >> v2:172.21.15.183:3300/0 conn(0x7f0fac0b1f30 0x7f0fac0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:40:42.746 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 >> v1:172.21.15.50:6789/0 conn(0x7f0fac06f270 legacy=0x7f0fac06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:42.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f94003970 0x7f0f94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f940047c0 0x7f0f94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f94005180 0x7f0f94014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac15d6e0 con 0x7f0f94005180 2023-10-19T10:40:42.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0b2850 con 0x7f0f94003970 2023-10-19T10:40:42.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94014860 con 0x7f0f940047c0 2023-10-19T10:40:42.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fab7fe640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fac160340 0x7f0fac1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:40:42.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f94005180 0x7f0f94014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fabfff640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f94003970 0x7f0f94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fab7fe640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f940047c0 0x7f0f94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f940047c0 msgr2=0x7f0f94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:40:42.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f940047c0 0x7f0f94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f94003970 msgr2=0x7f0f94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:40:42.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f94003970 0x7f0f94003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1081180131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94003520 con 0x7f0f94005180 2023-10-19T10:40:42.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.738+0000 7f0fabfff640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f94003970 0x7f0f94003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:40:42.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.739+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f94005180 0x7f0f94014190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400a860 tx=0x7f0f940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06f6835c28b27e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:40:42.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.739+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4034630 con 0x7f0f94005180 2023-10-19T10:40:42.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.739+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa402f070 con 0x7f0f94005180 2023-10-19T10:40:42.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.739+0000 7f0faaffd640 1 -- 172.21.15.50:0/1081180131 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40349f0 con 0x7f0f94005180 2023-10-19T10:40:42.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb2641640 1 -- 172.21.15.50:0/1081180131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f94005180 msgr2=0x7f0f94014190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:42.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1081180131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f94005180 0x7f0f94014190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400a860 tx=0x7f0f940057f0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1081180131 reap_dead start 2023-10-19T10:40:42.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb2641640 1 -- 172.21.15.50:0/1081180131 shutdown_connections 2023-10-19T10:40:42.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb2641640 1 -- 172.21.15.50:0/1081180131 >> 172.21.15.50:0/1081180131 conn(0x7f0fac0ccf30 msgr2=0x7f0fac0cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:40:42.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb2641640 1 -- 172.21.15.50:0/1081180131 shutdown_connections 2023-10-19T10:40:42.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.740+0000 7f0fb2641640 1 -- 172.21.15.50:0/1081180131 wait complete. 2023-10-19T10:40:42.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.741+0000 7f0fb2641640 1 Processor -- start 2023-10-19T10:40:42.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.741+0000 7f0fb2641640 1 -- start start 2023-10-19T10:40:42.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb2641640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f94005180 0x7f0fac154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb2641640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fac15f580 0x7f0fac157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb2641640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 0x7f0fac0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb2641640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac157540 con 0x7f0fac1580f0 2023-10-19T10:40:42.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb2641640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac156950 con 0x7f0fac15f580 2023-10-19T10:40:42.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb2641640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac156ba0 con 0x7f0f94005180 2023-10-19T10:40:42.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 0x7f0fac0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 0x7f0fac0b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41986/0 (socket says 172.21.15.50:41986) 2023-10-19T10:40:42.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1736082826 learned_addr learned my addr 172.21.15.50:0/1736082826 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:40:42.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fab7fe640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fac15f580 0x7f0fac157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f94005180 msgr2=0x7f0fac154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:42.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f94005180 0x7f0fac154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fac15f580 msgr2=0x7f0fac157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:42.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fac15f580 0x7f0fac157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 -- 172.21.15.50:0/1736082826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac156e80 con 0x7f0fac1580f0 2023-10-19T10:40:42.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fb0bde640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 0x7f0fac0b3070 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400a860 tx=0x7f0fa4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:40:42.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.742+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4001fd0 con 0x7f0fac1580f0 2023-10-19T10:40:42.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.743+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa4034640 con 0x7f0fac1580f0 2023-10-19T10:40:42.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.743+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac157160 con 0x7f0fac1580f0 2023-10-19T10:40:42.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.743+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40349b0 con 0x7f0fac1580f0 2023-10-19T10:40:42.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.743+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac0b4730 con 0x7f0fac1580f0 2023-10-19T10:40:42.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.744+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac0729f0 con 0x7f0fac1580f0 2023-10-19T10:40:42.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.744+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0fa4031030 con 0x7f0fac1580f0 2023-10-19T10:40:42.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.744+0000 7f0fa8ff9640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0f9806a080 0x7f0f9806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:42.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.745+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0fa40aa7a0 con 0x7f0fac1580f0 2023-10-19T10:40:42.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.745+0000 7f0fabfff640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0f9806a080 0x7f0f9806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:42.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.745+0000 7f0fabfff640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0f9806a080 0x7f0f9806c540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006fd0 tx=0x7f0f9c00c2d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:40:42.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.749+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0fa4079830 con 0x7f0fac1580f0 2023-10-19T10:40:42.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.958+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fac0729f0 con 0x7f0fac1580f0 2023-10-19T10:40:42.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.959+0000 7f0fa8ff9640 1 -- 172.21.15.50:0/1736082826 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f0fa40793c0 con 0x7f0fac1580f0 2023-10-19T10:40:42.962 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:40:42.962 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:40:42.962 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:40:42.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0f9806a080 msgr2=0x7f0f9806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:42.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0f9806a080 0x7f0f9806c540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006fd0 tx=0x7f0f9c00c2d0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 msgr2=0x7f0fac0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:42.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 0x7f0fac0b3070 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400a860 tx=0x7f0fa4001d80 comp rx=0 tx=0).stop 2023-10-19T10:40:42.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 shutdown_connections 2023-10-19T10:40:42.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0f9806a080 0x7f0f9806c540 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fac1580f0 0x7f0fac0b3070 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fac15f580 0x7f0fac157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 --2- 172.21.15.50:0/1736082826 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f94005180 0x7f0fac154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:42.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.960+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 >> 172.21.15.50:0/1736082826 conn(0x7f0fac0ccf30 msgr2=0x7f0fac06b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:40:42.965 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.961+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 shutdown_connections 2023-10-19T10:40:42.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:42.961+0000 7f0fb2641640 1 -- 172.21.15.50:0/1736082826 wait complete. 2023-10-19T10:40:42.974 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:40:43.096 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":122.630906497,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.75327069606025487,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":124.219866404,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.75051030091830806,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:40:43.097 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.7532706960602549 }, "state": "open", "uptime": 122.630906497 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.7505103009183081 }, "state": "open", "uptime": 124.219866404 } ] 2023-10-19T10:40:43.097 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.202+0000 7fb1db56e640 1 Processor -- start 2023-10-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.202+0000 7fb1db56e640 1 -- start start 2023-10-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1db56e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb1d40d0e30 0x7fb1d415c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1db56e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb1d40cd920 0x7fb1d40d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1db56e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d406aa30 con 0x7fb1d415c570 2023-10-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1db56e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d415ce90 con 0x7fb1d40d0e30 2023-10-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1db56e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d406b2d0 con 0x7fb1d40cd920 2023-10-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1d8b09640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb1d40d0e30 0x7fb1d415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1d930a640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb1d415c570 0x7fb1d406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:48650/0 (socket says 172.21.15.50:48650) 2023-10-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.203+0000 7fb1d930a640 1 -- 172.21.15.50:0/3346053321 learned_addr learned my addr 172.21.15.50:0/3346053321 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.204+0000 7fb1d9b0b640 1 --2- 172.21.15.50:0/3346053321 >> v2:172.21.15.183:3300/0 conn(0x7fb1d40cd920 0x7fb1d40d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.204+0000 7fb1d8b09640 1 -- 172.21.15.50:0/3346053321 >> v2:172.21.15.183:3300/0 conn(0x7fb1d40cd920 msgr2=0x7fb1d40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.204+0000 7fb1d8b09640 1 --2- 172.21.15.50:0/3346053321 >> v2:172.21.15.183:3300/0 conn(0x7fb1d40cd920 0x7fb1d40d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.204+0000 7fb1d8b09640 1 -- 172.21.15.50:0/3346053321 >> v1:172.21.15.50:6789/0 conn(0x7fb1d415c570 legacy=0x7fb1d406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:43.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.204+0000 7fb1d3fff640 1 -- 172.21.15.50:0/3346053321 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541001055 0 0) 0x7fb1d406aa30 con 0x7fb1d415c570 2023-10-19T10:40:43.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.204+0000 7fb1d8b09640 1 -- 172.21.15.50:0/3346053321 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40d1270 con 0x7fb1d40d0e30 2023-10-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1d8b09640 1 --2- 172.21.15.50:0/3346053321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d415c950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40d3770 tx=0x7fb1c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1692309557d241 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1d3fff640 1 -- 172.21.15.50:0/3346053321 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c802e070 con 0x7fb1d40d0e30 2023-10-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1d3fff640 1 -- 172.21.15.50:0/3346053321 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c80315a0 con 0x7fb1d40d0e30 2023-10-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1d3fff640 1 -- 172.21.15.50:0/3346053321 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8031930 con 0x7fb1d40d0e30 2023-10-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 -- 172.21.15.50:0/3346053321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 msgr2=0x7fb1d415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 --2- 172.21.15.50:0/3346053321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d415c950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40d3770 tx=0x7fb1c802b840 comp rx=0 tx=0).stop 2023-10-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 -- 172.21.15.50:0/3346053321 shutdown_connections 2023-10-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 --2- 172.21.15.50:0/3346053321 >> v2:172.21.15.183:3300/0 conn(0x7fb1d40cd920 0x7fb1d40d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 --2- 172.21.15.50:0/3346053321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d415c950 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 -- 172.21.15.50:0/3346053321 >> 172.21.15.50:0/3346053321 conn(0x7fb1d4089ff0 msgr2=0x7fb1d409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 -- 172.21.15.50:0/3346053321 shutdown_connections 2023-10-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.205+0000 7fb1db56e640 1 -- 172.21.15.50:0/3346053321 wait complete. 2023-10-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 Processor -- start 2023-10-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 -- start start 2023-10-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1d40cd920 0x7fb1d414ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d40c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1d415c570 0x7fb1d40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40c77a0 con 0x7fb1d40d0e30 2023-10-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40c6b90 con 0x7fb1d40cd920 2023-10-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1db56e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40c6e70 con 0x7fb1d415c570 2023-10-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1d8b09640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d40c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1d8b09640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d40c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41998/0 (socket says 172.21.15.50:41998) 2023-10-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.207+0000 7fb1d8b09640 1 -- 172.21.15.50:0/873443207 learned_addr learned my addr 172.21.15.50:0/873443207 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d930a640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1d40cd920 0x7fb1d414ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d9b0b640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1d415c570 0x7fb1d40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d8b09640 1 -- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1d415c570 msgr2=0x7fb1d40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d8b09640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1d415c570 0x7fb1d40b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d8b09640 1 -- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1d40cd920 msgr2=0x7fb1d414ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d8b09640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1d40cd920 0x7fb1d414ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d8b09640 1 -- 172.21.15.50:0/873443207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c8009400 con 0x7fb1d40d0e30 2023-10-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d8b09640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d40c6240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8034040 tx=0x7fb1c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8004230 con 0x7fb1d40d0e30 2023-10-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8031f00 con 0x7fb1d40d0e30 2023-10-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d40c7150 con 0x7fb1d40d0e30 2023-10-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8032640 con 0x7fb1d40d0e30 2023-10-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.208+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d414b280 con 0x7fb1d40d0e30 2023-10-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.209+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d406af40 con 0x7fb1d40d0e30 2023-10-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.210+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1c80327a0 con 0x7fb1d40d0e30 2023-10-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.210+0000 7fb1d1ffb640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1c006a050 0x7fb1c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.210+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb1c80aa7d0 con 0x7fb1d40d0e30 2023-10-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.210+0000 7fb1d930a640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1c006a050 0x7fb1c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.211+0000 7fb1d930a640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1c006a050 0x7fb1c006c510 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc001460 tx=0x7fb1cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.214+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb1d406af40 con 0x7fb1d40d0e30 2023-10-19T10:40:43.268 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:40:43.265+0000 7f8c5aa63640 -1 osd.2 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:40:43.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.427+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d406af40 con 0x7fb1d40d0e30 2023-10-19T10:40:43.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.427+0000 7fb1d1ffb640 1 -- 172.21.15.50:0/873443207 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7fb1d406af40 con 0x7fb1d40d0e30 2023-10-19T10:40:43.430 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:40:43.431 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:40:43.431 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:40:43.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1c006a050 msgr2=0x7fb1c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:43.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1c006a050 0x7fb1c006c510 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc001460 tx=0x7fb1cc009040 comp rx=0 tx=0).stop 2023-10-19T10:40:43.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 msgr2=0x7fb1d40c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:40:43.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d40c6240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8034040 tx=0x7fb1c80046c0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 shutdown_connections 2023-10-19T10:40:43.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb1c006a050 0x7fb1c006c510 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1d415c570 0x7fb1d40b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d40d0e30 0x7fb1d40c6240 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 --2- 172.21.15.50:0/873443207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1d40cd920 0x7fb1d414ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:40:43.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 >> 172.21.15.50:0/873443207 conn(0x7fb1d4089ff0 msgr2=0x7fb1d4072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:40:43.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 shutdown_connections 2023-10-19T10:40:43.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:40:43.430+0000 7fb1db56e640 1 -- 172.21.15.50:0/873443207 wait complete. 2023-10-19T10:40:56.272 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:40:56.276 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:41:26.343 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:41:26.347 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:41:56.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:41:56.415 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:42:13.443 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:42:13.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.560+0000 7f46148bd640 1 Processor -- start 2023-10-19T10:42:13.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.560+0000 7f46148bd640 1 -- start start 2023-10-19T10:42:13.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f46148bd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f46100d0e30 0x7f461015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:13.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f46148bd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46100cd920 0x7f46100d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:13.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f46148bd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461006aa30 con 0x7f461015c570 2023-10-19T10:42:13.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f46148bd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461015ce90 con 0x7f46100d0e30 2023-10-19T10:42:13.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f46148bd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461006b2d0 con 0x7f46100cd920 2023-10-19T10:42:13.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f460dd9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f46100d0e30 0x7f461015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:13.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f460dd9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f46100d0e30 0x7f461015c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53718/0 (socket says 172.21.15.50:53718) 2023-10-19T10:42:13.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f460dd9b640 1 -- 172.21.15.50:0/2752341896 learned_addr learned my addr 172.21.15.50:0/2752341896 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:42:13.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.561+0000 7f460ed9d640 1 --2- 172.21.15.50:0/2752341896 >> v2:172.21.15.183:3300/0 conn(0x7f46100cd920 0x7f46100d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:13.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.562+0000 7f460dd9b640 1 -- 172.21.15.50:0/2752341896 >> v2:172.21.15.183:3300/0 conn(0x7f46100cd920 msgr2=0x7f46100d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:13.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.562+0000 7f460dd9b640 1 --2- 172.21.15.50:0/2752341896 >> v2:172.21.15.183:3300/0 conn(0x7f46100cd920 0x7f46100d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.562+0000 7f460dd9b640 1 -- 172.21.15.50:0/2752341896 >> v1:172.21.15.183:6789/0 conn(0x7f461015c570 legacy=0x7f461006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:13.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.562+0000 7f460dd9b640 1 -- 172.21.15.50:0/2752341896 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100d1270 con 0x7f46100d0e30 2023-10-19T10:42:13.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.562+0000 7f460dd9b640 1 --2- 172.21.15.50:0/2752341896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100d0e30 0x7f461015c950 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f46100d3770 tx=0x7f460402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6c7e7df2176834 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:13.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.562+0000 7f460d59a640 1 -- 172.21.15.50:0/2752341896 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604031870 con 0x7f46100d0e30 2023-10-19T10:42:13.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f460d59a640 1 -- 172.21.15.50:0/2752341896 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040319d0 con 0x7f46100d0e30 2023-10-19T10:42:13.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f460d59a640 1 -- 172.21.15.50:0/2752341896 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604031d60 con 0x7f46100d0e30 2023-10-19T10:42:13.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 -- 172.21.15.50:0/2752341896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100d0e30 msgr2=0x7f461015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:13.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 --2- 172.21.15.50:0/2752341896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100d0e30 0x7f461015c950 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f46100d3770 tx=0x7f460402b910 comp rx=0 tx=0).stop 2023-10-19T10:42:13.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 -- 172.21.15.50:0/2752341896 shutdown_connections 2023-10-19T10:42:13.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 --2- 172.21.15.50:0/2752341896 >> v2:172.21.15.183:3300/0 conn(0x7f46100cd920 0x7f46100d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 --2- 172.21.15.50:0/2752341896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100d0e30 0x7f461015c950 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 -- 172.21.15.50:0/2752341896 >> 172.21.15.50:0/2752341896 conn(0x7f4610089ff0 msgr2=0x7f461009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:42:13.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 -- 172.21.15.50:0/2752341896 shutdown_connections 2023-10-19T10:42:13.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.563+0000 7f46148bd640 1 -- 172.21.15.50:0/2752341896 wait complete. 2023-10-19T10:42:13.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.564+0000 7f46148bd640 1 Processor -- start 2023-10-19T10:42:13.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 -- start start 2023-10-19T10:42:13.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 0x7f461014c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:13.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46100d0e30 0x7f461015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:13.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f461015c570 0x7f46101f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:13.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100c97e0 con 0x7f46100cd920 2023-10-19T10:42:13.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461015f690 con 0x7f461015c570 2023-10-19T10:42:13.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.565+0000 7f46148bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461015f970 con 0x7f46100d0e30 2023-10-19T10:42:13.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 0x7f461014c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:13.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 0x7f461014c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53728/0 (socket says 172.21.15.50:53728) 2023-10-19T10:42:13.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 -- 172.21.15.50:0/1683388384 learned_addr learned my addr 172.21.15.50:0/1683388384 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:42:13.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460ed9d640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f461015c570 0x7f46101f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:13.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460dd9b640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46100d0e30 0x7f461015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:13.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 -- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46100d0e30 msgr2=0x7f461015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:13.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46100d0e30 0x7f461015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 -- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f461015c570 msgr2=0x7f46101f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:13.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f461015c570 0x7f46101f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 -- 172.21.15.50:0/1683388384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040094d0 con 0x7f46100cd920 2023-10-19T10:42:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f460e59c640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 0x7f461014c900 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f460000c450 tx=0x7f460000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46000052d0 con 0x7f46100cd920 2023-10-19T10:42:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600005450 con 0x7f46100cd920 2023-10-19T10:42:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461015fc50 con 0x7f46100cd920 2023-10-19T10:42:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003880 con 0x7f46100cd920 2023-10-19T10:42:13.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.566+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46101f2800 con 0x7f46100cd920 2023-10-19T10:42:13.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.567+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461015fe40 con 0x7f46100cd920 2023-10-19T10:42:13.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.568+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f46000039e0 con 0x7f46100cd920 2023-10-19T10:42:13.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.568+0000 7f45feffd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f45f006a050 0x7f45f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:13.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.568+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4600088160 con 0x7f46100cd920 2023-10-19T10:42:13.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.569+0000 7f460dd9b640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f45f006a050 0x7f45f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:13.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.569+0000 7f460dd9b640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f45f006a050 0x7f45f006c510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4604036000 tx=0x7f460403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:13.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.573+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f4600053780 con 0x7f46100cd920 2023-10-19T10:42:13.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.785+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46101f2990 con 0x7f46100cd920 2023-10-19T10:42:13.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.785+0000 7f45feffd640 1 -- 172.21.15.50:0/1683388384 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f4600056da0 con 0x7f46100cd920 2023-10-19T10:42:13.789 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:42:13.789 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:42:13.789 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:42:13.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f45f006a050 msgr2=0x7f45f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:13.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f45f006a050 0x7f45f006c510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4604036000 tx=0x7f460403c040 comp rx=0 tx=0).stop 2023-10-19T10:42:13.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 msgr2=0x7f461014c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:13.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 0x7f461014c900 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f460000c450 tx=0x7f460000c910 comp rx=0 tx=0).stop 2023-10-19T10:42:13.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 shutdown_connections 2023-10-19T10:42:13.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f45f006a050 0x7f45f006c510 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f461015c570 0x7f46101f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46100d0e30 0x7f461015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 --2- 172.21.15.50:0/1683388384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46100cd920 0x7f461014c900 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:13.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 >> 172.21.15.50:0/1683388384 conn(0x7f4610089ff0 msgr2=0x7f46100b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:42:13.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 shutdown_connections 2023-10-19T10:42:13.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:13.787+0000 7f46148bd640 1 -- 172.21.15.50:0/1683388384 wait complete. 2023-10-19T10:42:13.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:42:13.923 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":213.45861378399999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.61067694590793775,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":215.047573691,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.60843909212761726,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:42:13.924 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6106769459079378 }, "state": "open", "uptime": 213.458613784 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6084390921276173 }, "state": "stale", "uptime": 215.047573691 } ] 2023-10-19T10:42:13.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:42:13.927 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:42:14.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.031+0000 7f259af70640 1 Processor -- start 2023-10-19T10:42:14.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.031+0000 7f259af70640 1 -- start start 2023-10-19T10:42:14.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f259af70640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f259406de70 0x7f25940cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f259af70640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 0x7f2594069f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f259af70640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25940ccab0 con 0x7f259406ef00 2023-10-19T10:42:14.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f259af70640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259415ef70 con 0x7f259406de70 2023-10-19T10:42:14.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f259af70640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259415e1b0 con 0x7f259406e250 2023-10-19T10:42:14.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f2593fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f259406ef00 0x7f2594069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:58548/0 (socket says 172.21.15.50:58548) 2023-10-19T10:42:14.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f2593fff640 1 -- 172.21.15.50:0/1518125006 learned_addr learned my addr 172.21.15.50:0/1518125006 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:42:14.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f2598d0c640 1 --2- 172.21.15.50:0/1518125006 >> v2:172.21.15.183:3300/0 conn(0x7f259406de70 0x7f25940cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f259950d640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 0x7f2594069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924606440 0 0) 0x7f25940ccab0 con 0x7f259406ef00 2023-10-19T10:42:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.032+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2584003520 con 0x7f259406ef00 2023-10-19T10:42:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1022132874 0 0) 0x7f2584003520 con 0x7f259406ef00 2023-10-19T10:42:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 msgr2=0x7f2594069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:14.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 0x7f2594069f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 >> v2:172.21.15.183:3300/0 conn(0x7f259406de70 msgr2=0x7f25940cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 --2- 172.21.15.50:0/1518125006 >> v2:172.21.15.183:3300/0 conn(0x7f259406de70 0x7f25940cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259415e490 con 0x7f259406ef00 2023-10-19T10:42:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f2580001670 con 0x7f259406ef00 2023-10-19T10:42:14.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 >> v1:172.21.15.50:6789/0 conn(0x7f259406ef00 legacy=0x7f2594069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 0x7f2584003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584003ef0 0x7f25840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2584004d00 0x7f2584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259415ef70 con 0x7f2584004d00 2023-10-19T10:42:14.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259415e1b0 con 0x7f2584003ef0 2023-10-19T10:42:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258400b9d0 con 0x7f259406e250 2023-10-19T10:42:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f259950d640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2584004d00 0x7f2584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f2598d0c640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 0x7f2584003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f2593fff640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584003ef0 0x7f25840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f259950d640 1 -- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 msgr2=0x7f2584003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f259950d640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f259406e250 0x7f2584003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f259950d640 1 -- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584003ef0 msgr2=0x7f25840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f259950d640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584003ef0 0x7f25840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.045 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.033+0000 7f259950d640 1 -- 172.21.15.50:0/1518125006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584003520 con 0x7f2584004d00 2023-10-19T10:42:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.034+0000 7f259950d640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2584004d00 0x7f2584005180 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e170 tx=0x7f25840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df557a234c4e373 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.034+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02cbf0 con 0x7f2584004d00 2023-10-19T10:42:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.034+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c00f040 con 0x7f2584004d00 2023-10-19T10:42:14.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.034+0000 7f25937fe640 1 -- 172.21.15.50:0/1518125006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c034440 con 0x7f2584004d00 2023-10-19T10:42:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259af70640 1 -- 172.21.15.50:0/1518125006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2584004d00 msgr2=0x7f2584005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259af70640 1 --2- 172.21.15.50:0/1518125006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2584004d00 0x7f2584005180 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e170 tx=0x7f25840057f0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259950d640 1 -- 172.21.15.50:0/1518125006 reap_dead start 2023-10-19T10:42:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259af70640 1 -- 172.21.15.50:0/1518125006 shutdown_connections 2023-10-19T10:42:14.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259af70640 1 -- 172.21.15.50:0/1518125006 >> 172.21.15.50:0/1518125006 conn(0x7f25940cbc00 msgr2=0x7f25940cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:42:14.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259af70640 1 -- 172.21.15.50:0/1518125006 shutdown_connections 2023-10-19T10:42:14.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.035+0000 7f259af70640 1 -- 172.21.15.50:0/1518125006 wait complete. 2023-10-19T10:42:14.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.036+0000 7f259af70640 1 Processor -- start 2023-10-19T10:42:14.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.036+0000 7f259af70640 1 -- start start 2023-10-19T10:42:14.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259af70640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2584004d00 0x7f259414da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259af70640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 0x7f25940cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259af70640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f259415c640 0x7f259414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259af70640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940c71f0 con 0x7f259414ec80 2023-10-19T10:42:14.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259af70640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259414f460 con 0x7f259415c640 2023-10-19T10:42:14.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259af70640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259414f180 con 0x7f2584004d00 2023-10-19T10:42:14.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f2593fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 0x7f25940cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f2593fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 0x7f25940cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53746/0 (socket says 172.21.15.50:53746) 2023-10-19T10:42:14.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f2593fff640 1 -- 172.21.15.50:0/3725698448 learned_addr learned my addr 172.21.15.50:0/3725698448 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:42:14.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f2598d0c640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2584004d00 0x7f259414da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.037+0000 7f259950d640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f259415c640 0x7f259414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f2593fff640 1 -- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2584004d00 msgr2=0x7f259414da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f2593fff640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2584004d00 0x7f259414da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f2593fff640 1 -- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f259415c640 msgr2=0x7f259414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f2593fff640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f259415c640 0x7f259414bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f2593fff640 1 -- 172.21.15.50:0/3725698448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940c7420 con 0x7f259414ec80 2023-10-19T10:42:14.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f2593fff640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 0x7f25940cc6f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2580007290 tx=0x7f2580007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:14.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000f960 con 0x7f259414ec80 2023-10-19T10:42:14.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258000fac0 con 0x7f259414ec80 2023-10-19T10:42:14.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25940c7700 con 0x7f259414ec80 2023-10-19T10:42:14.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000fe70 con 0x7f259414ec80 2023-10-19T10:42:14.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.038+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259414df40 con 0x7f259414ec80 2023-10-19T10:42:14.057 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.039+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25940c7c20 con 0x7f259414ec80 2023-10-19T10:42:14.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.040+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f258001d460 con 0x7f259414ec80 2023-10-19T10:42:14.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.040+0000 7f25917fa640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f257806a050 0x7f257806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.058 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.041+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2580088900 con 0x7f259414ec80 2023-10-19T10:42:14.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.041+0000 7f2598d0c640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f257806a050 0x7f257806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.041+0000 7f2598d0c640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f257806a050 0x7f257806c510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2588009c40 tx=0x7f2588009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:14.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.045+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f25940c7c20 con 0x7f259414ec80 2023-10-19T10:42:14.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.055+0000 7ff28e1cb640 1 Processor -- start 2023-10-19T10:42:14.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.055+0000 7ff28e1cb640 1 -- start start 2023-10-19T10:42:14.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff28e1cb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff28815c680 0x7ff2880d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff28e1cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880cd960 0x7ff2880d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff28e1cb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2880d0e70 0x7ff2880ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff28e1cb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880d1300 con 0x7ff28815c680 2023-10-19T10:42:14.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff28e1cb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880cde90 con 0x7ff2880d0e70 2023-10-19T10:42:14.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff28e1cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880ce170 con 0x7ff2880cd960 2023-10-19T10:42:14.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff28815c680 0x7ff2880d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff28815c680 0x7ff2880d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53754/0 (socket says 172.21.15.50:53754) 2023-10-19T10:42:14.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 -- 172.21.15.50:0/2703742902 learned_addr learned my addr 172.21.15.50:0/2703742902 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:42:14.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff286ffd640 1 --2- 172.21.15.50:0/2703742902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880cd960 0x7ff2880d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff287fff640 1 --2- 172.21.15.50:0/2703742902 >> v2:172.21.15.183:3300/0 conn(0x7ff2880d0e70 0x7ff2880ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 -- 172.21.15.50:0/2703742902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880cd960 msgr2=0x7ff2880d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:14.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 --2- 172.21.15.50:0/2703742902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880cd960 0x7ff2880d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 -- 172.21.15.50:0/2703742902 >> v2:172.21.15.183:3300/0 conn(0x7ff2880d0e70 msgr2=0x7ff2880ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:42:14.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 --2- 172.21.15.50:0/2703742902 >> v2:172.21.15.183:3300/0 conn(0x7ff2880d0e70 0x7ff2880ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.056+0000 7ff2877fe640 1 -- 172.21.15.50:0/2703742902 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880b1c80 con 0x7ff28815c680 2023-10-19T10:42:14.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.057+0000 7ff2877fe640 1 --2- 172.21.15.50:0/2703742902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff28815c680 0x7ff2880d0090 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff2880cdda0 tx=0x7ff27802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f308fac958f1662d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:14.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.057+0000 7ff2867fc640 1 -- 172.21.15.50:0/2703742902 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278004230 con 0x7ff28815c680 2023-10-19T10:42:14.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.057+0000 7ff2867fc640 1 -- 172.21.15.50:0/2703742902 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278004390 con 0x7ff28815c680 2023-10-19T10:42:14.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.057+0000 7ff2867fc640 1 -- 172.21.15.50:0/2703742902 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278030840 con 0x7ff28815c680 2023-10-19T10:42:14.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 -- 172.21.15.50:0/2703742902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff28815c680 msgr2=0x7ff2880d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 --2- 172.21.15.50:0/2703742902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff28815c680 0x7ff2880d0090 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff2880cdda0 tx=0x7ff27802b980 comp rx=0 tx=0).stop 2023-10-19T10:42:14.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 -- 172.21.15.50:0/2703742902 shutdown_connections 2023-10-19T10:42:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 --2- 172.21.15.50:0/2703742902 >> v2:172.21.15.183:3300/0 conn(0x7ff2880d0e70 0x7ff2880ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 --2- 172.21.15.50:0/2703742902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880cd960 0x7ff2880d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 --2- 172.21.15.50:0/2703742902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff28815c680 0x7ff2880d0090 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 -- 172.21.15.50:0/2703742902 >> 172.21.15.50:0/2703742902 conn(0x7ff2880d2370 msgr2=0x7ff2880d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:42:14.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 -- 172.21.15.50:0/2703742902 shutdown_connections 2023-10-19T10:42:14.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.058+0000 7ff28e1cb640 1 -- 172.21.15.50:0/2703742902 wait complete. 2023-10-19T10:42:14.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.059+0000 7ff28e1cb640 1 Processor -- start 2023-10-19T10:42:14.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.059+0000 7ff28e1cb640 1 -- start start 2023-10-19T10:42:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff28e1cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2880cd960 0x7ff288159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff28e1cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2880d0e70 0x7ff28815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff28e1cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff28815c680 0x7ff28815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff28e1cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28815a350 con 0x7ff2880cd960 2023-10-19T10:42:14.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff28e1cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880c58a0 con 0x7ff2880d0e70 2023-10-19T10:42:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff28e1cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288158d70 con 0x7ff28815c680 2023-10-19T10:42:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2880cd960 0x7ff288159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2880cd960 0x7ff288159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53770/0 (socket says 172.21.15.50:53770) 2023-10-19T10:42:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 -- 172.21.15.50:0/567288555 learned_addr learned my addr 172.21.15.50:0/567288555 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:42:14.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 -- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff28815c680 msgr2=0x7ff28815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff287fff640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff28815c680 0x7ff28815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff286ffd640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2880d0e70 0x7ff28815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff28815c680 0x7ff28815f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 -- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2880d0e70 msgr2=0x7ff28815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2880d0e70 0x7ff28815cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.060+0000 7ff2877fe640 1 -- 172.21.15.50:0/567288555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009460 con 0x7ff2880cd960 2023-10-19T10:42:14.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff2877fe640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2880cd960 0x7ff288159e10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff2780090b0 tx=0x7ff2780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:14.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff26ffff640 1 -- 172.21.15.50:0/567288555 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27803d070 con 0x7ff2880cd960 2023-10-19T10:42:14.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff26ffff640 1 -- 172.21.15.50:0/567288555 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278004500 con 0x7ff2880cd960 2023-10-19T10:42:14.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff28e1cb640 1 -- 172.21.15.50:0/567288555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff288159050 con 0x7ff2880cd960 2023-10-19T10:42:14.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff26ffff640 1 -- 172.21.15.50:0/567288555 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278030e00 con 0x7ff2880cd960 2023-10-19T10:42:14.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff28e1cb640 1 -- 172.21.15.50:0/567288555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff288159610 con 0x7ff2880cd960 2023-10-19T10:42:14.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.061+0000 7ff28e1cb640 1 -- 172.21.15.50:0/567288555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff288155230 con 0x7ff2880cd960 2023-10-19T10:42:14.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.062+0000 7ff26ffff640 1 -- 172.21.15.50:0/567288555 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff278003610 con 0x7ff2880cd960 2023-10-19T10:42:14.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.063+0000 7ff26ffff640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27406a050 0x7ff27406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:42:14.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.063+0000 7ff26ffff640 1 -- 172.21.15.50:0/567288555 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff2780aaec0 con 0x7ff2880cd960 2023-10-19T10:42:14.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.063+0000 7ff26ffff640 1 -- 172.21.15.50:0/567288555 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+447 (secure 0 0 0) 0x7ff2780ab330 con 0x7ff2880cd960 2023-10-19T10:42:14.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.063+0000 7ff286ffd640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27406a050 0x7ff27406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:42:14.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.063+0000 7ff28e1cb640 1 -- 172.21.15.50:0/567288555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff288066e80 con 0x7ff2880cd960 2023-10-19T10:42:14.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.064+0000 7ff286ffd640 1 --2- 172.21.15.50:0/567288555 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff27406a050 0x7ff27406c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2881e6fc0 tx=0x7ff27c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:42:14.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.259+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25940c7c20 con 0x7f259414ec80 2023-10-19T10:42:14.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.260+0000 7f25917fa640 1 -- 172.21.15.50:0/3725698448 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f25940c7c20 con 0x7f259414ec80 2023-10-19T10:42:14.263 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:42:14.263 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:42:14.263 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:42:14.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f257806a050 msgr2=0x7f257806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f257806a050 0x7f257806c510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2588009c40 tx=0x7f2588009190 comp rx=0 tx=0).stop 2023-10-19T10:42:14.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 msgr2=0x7f25940cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:42:14.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 0x7f25940cc6f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2580007290 tx=0x7f2580007690 comp rx=0 tx=0).stop 2023-10-19T10:42:14.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 shutdown_connections 2023-10-19T10:42:14.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f257806a050 0x7f257806c510 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f259415c640 0x7f259414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f259414ec80 0x7f25940cc6f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 --2- 172.21.15.50:0/3725698448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2584004d00 0x7f259414da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:42:14.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 >> 172.21.15.50:0/3725698448 conn(0x7f25940cbc00 msgr2=0x7f259415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:42:14.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 shutdown_connections 2023-10-19T10:42:14.266 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:42:14.262+0000 7f259af70640 1 -- 172.21.15.50:0/3725698448 wait complete. 2023-10-19T10:42:14.275 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:42:14.398 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":213.93260180199999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.61000853226059404,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":215.521561709,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.60777312791285421,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:42:14.399 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.610008532260594 }, "state": "open", "uptime": 213.932601802 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6077731279128542 }, "state": "stale", "uptime": 215.521561709 } ] 2023-10-19T10:42:14.500 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:42:26.718 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:42:26.722 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:42:56.840 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:42:56.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:43:14.513 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:43:14.514 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: '1 client(s) laggy due to laggy OSDs', retrying 2023-10-19T10:43:15.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:43:15.519 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:43:15.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.636+0000 7f6d0eba6640 1 Processor -- start 2023-10-19T10:43:15.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.636+0000 7f6d0eba6640 1 -- start start 2023-10-19T10:43:15.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.636+0000 7f6d0eba6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6d080caf90 0x7f6d08159210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.636+0000 7f6d0eba6640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080c95d0 con 0x7f6d08159750 2023-10-19T10:43:15.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.636+0000 7f6d0eba6640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080c8ca0 con 0x7f6d08156510 2023-10-19T10:43:15.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.636+0000 7f6d0eba6640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0806f770 con 0x7f6d080caf90 2023-10-19T10:43:15.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.637+0000 7f6d07fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f6d080caf90 0x7f6d08159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.637+0000 7f6d0d143640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f6d08159750 0x7f6d080cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39050/0 (socket says 172.21.15.50:39050) 2023-10-19T10:43:15.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.637+0000 7f6d0d143640 1 -- 172.21.15.50:0/505364535 learned_addr learned my addr 172.21.15.50:0/505364535 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:43:15.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.637+0000 7f6d07fff640 1 -- 172.21.15.50:0/505364535 >> v1:172.21.15.183:6789/0 conn(0x7f6d08156510 legacy=0x7f6d080ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.637+0000 7f6d07fff640 1 -- 172.21.15.50:0/505364535 >> v1:172.21.15.50:6789/0 conn(0x7f6d08159750 legacy=0x7f6d080cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.637+0000 7f6d07fff640 1 -- 172.21.15.50:0/505364535 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080c8f80 con 0x7f6d080caf90 2023-10-19T10:43:15.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.638+0000 7f6d07fff640 1 --2- 172.21.15.50:0/505364535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d080caf90 0x7f6d08159210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40097a0 tx=0x7f6cf402ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f710f4f51843fd62 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:43:15.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.638+0000 7f6d077fe640 1 -- 172.21.15.50:0/505364535 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf402e830 con 0x7f6d080caf90 2023-10-19T10:43:15.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.638+0000 7f6d077fe640 1 -- 172.21.15.50:0/505364535 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf402e990 con 0x7f6d080caf90 2023-10-19T10:43:15.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.638+0000 7f6d077fe640 1 -- 172.21.15.50:0/505364535 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf402ed20 con 0x7f6d080caf90 2023-10-19T10:43:15.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 -- 172.21.15.50:0/505364535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d080caf90 msgr2=0x7f6d08159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/505364535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d080caf90 0x7f6d08159210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40097a0 tx=0x7f6cf402ca70 comp rx=0 tx=0).stop 2023-10-19T10:43:15.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 -- 172.21.15.50:0/505364535 shutdown_connections 2023-10-19T10:43:15.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/505364535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d080caf90 0x7f6d08159210 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 -- 172.21.15.50:0/505364535 >> 172.21.15.50:0/505364535 conn(0x7f6d08069e90 msgr2=0x7f6d080afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:43:15.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 -- 172.21.15.50:0/505364535 shutdown_connections 2023-10-19T10:43:15.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.639+0000 7f6d0eba6640 1 -- 172.21.15.50:0/505364535 wait complete. 2023-10-19T10:43:15.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.640+0000 7f6d0eba6640 1 Processor -- start 2023-10-19T10:43:15.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.640+0000 7f6d0eba6640 1 -- start start 2023-10-19T10:43:15.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0eba6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080caf90 0x7f6d080c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0eba6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 0x7f6d0814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0eba6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d08159750 0x7f6d08150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0eba6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0814ebb0 con 0x7f6d08156510 2023-10-19T10:43:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0eba6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0814ff00 con 0x7f6d08159750 2023-10-19T10:43:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0eba6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0814fc20 con 0x7f6d080caf90 2023-10-19T10:43:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 0x7f6d0814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 0x7f6d0814f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58470/0 (socket says 172.21.15.50:58470) 2023-10-19T10:43:15.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 -- 172.21.15.50:0/1587622172 learned_addr learned my addr 172.21.15.50:0/1587622172 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:43:15.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0d143640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d08159750 0x7f6d08150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0c942640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080caf90 0x7f6d080c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 -- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080caf90 msgr2=0x7f6d080c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080caf90 0x7f6d080c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 -- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d08159750 msgr2=0x7f6d08150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d08159750 0x7f6d08150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d07fff640 1 -- 172.21.15.50:0/1587622172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4009400 con 0x7f6d08156510 2023-10-19T10:43:15.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.641+0000 7f6d0d143640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d08159750 0x7f6d08150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:43:15.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.642+0000 7f6d07fff640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 0x7f6d0814f040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6cf402d080 tx=0x7f6cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:43:15.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.642+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf403d070 con 0x7f6d08156510 2023-10-19T10:43:15.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.642+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf4038a10 con 0x7f6d08156510 2023-10-19T10:43:15.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.642+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0814e560 con 0x7f6d08156510 2023-10-19T10:43:15.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.642+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4038da0 con 0x7f6d08156510 2023-10-19T10:43:15.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.642+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0814f580 con 0x7f6d08156510 2023-10-19T10:43:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.643+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0814ed60 con 0x7f6d08156510 2023-10-19T10:43:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.643+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6cf404a400 con 0x7f6d08156510 2023-10-19T10:43:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.644+0000 7f6d057fa640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6cec06a050 0x7f6cec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.644+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6cf40aa560 con 0x7f6d08156510 2023-10-19T10:43:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.644+0000 7f6d0c942640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6cec06a050 0x7f6cec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.645+0000 7f6d0c942640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6cec06a050 0x7f6cec06c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000f60 tx=0x7f6cfc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:43:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.649+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6cf4075b60 con 0x7f6d08156510 2023-10-19T10:43:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.661+0000 7f239a318640 1 Processor -- start 2023-10-19T10:43:15.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.661+0000 7f239a318640 1 -- start start 2023-10-19T10:43:15.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f239a318640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 0x7f23940d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f239a318640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23940cde90 con 0x7f23940d1040 2023-10-19T10:43:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f239a318640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23940ce170 con 0x7f23940cd960 2023-10-19T10:43:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f239a318640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23940b1c80 con 0x7f239415c680 2023-10-19T10:43:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f23988b5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f23940d1040 0x7f23940b2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39062/0 (socket says 172.21.15.50:39062) 2023-10-19T10:43:15.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f23988b5640 1 -- 172.21.15.50:0/2449753478 learned_addr learned my addr 172.21.15.50:0/2449753478 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:43:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2393fff640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 0x7f23940d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835596468 0 0) 0x7f23940cde90 con 0x7f23940d1040 2023-10-19T10:43:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003520 con 0x7f23940d1040 2023-10-19T10:43:15.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596959309 0 0) 0x7f2380003520 con 0x7f23940d1040 2023-10-19T10:43:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 msgr2=0x7f23940d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:43:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 0x7f23940d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 >> v1:172.21.15.183:6789/0 conn(0x7f23940cd960 legacy=0x7f23940d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940b1f60 con 0x7f23940d1040 2023-10-19T10:43:15.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.662+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f238c0019f0 con 0x7f23940d1040 2023-10-19T10:43:15.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 >> v1:172.21.15.50:6789/0 conn(0x7f23940d1040 legacy=0x7f23940b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23800039a0 0x7f2380003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 0x7f23800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2380004d00 0x7f2380005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23940b1c80 con 0x7f23800039a0 2023-10-19T10:43:15.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238000ba30 con 0x7f2380004d00 2023-10-19T10:43:15.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238000bba0 con 0x7f239415c680 2023-10-19T10:43:15.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2393fff640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23800039a0 0x7f2380003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f23937fe640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 0x7f23800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f23988b5640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2380004d00 0x7f2380005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2393fff640 1 -- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 msgr2=0x7f23800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:43:15.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2393fff640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f239415c680 0x7f23800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2393fff640 1 -- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2380004d00 msgr2=0x7f2380005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:43:15.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2393fff640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2380004d00 0x7f2380005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f2393fff640 1 -- 172.21.15.50:0/2449753478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380003520 con 0x7f23800039a0 2023-10-19T10:43:15.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.663+0000 7f23988b5640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2380004d00 0x7f2380005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:43:15.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.664+0000 7f2393fff640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23800039a0 0x7f2380003d80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2384009ab0 tx=0x7f23800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f78d14f6c0a9e9a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:43:15.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384005990 con 0x7f23800039a0 2023-10-19T10:43:15.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2384005af0 con 0x7f23800039a0 2023-10-19T10:43:15.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f2392ffd640 1 -- 172.21.15.50:0/2449753478 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384005ea0 con 0x7f23800039a0 2023-10-19T10:43:15.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f239a318640 1 -- 172.21.15.50:0/2449753478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23800039a0 msgr2=0x7f2380003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f239a318640 1 --2- 172.21.15.50:0/2449753478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23800039a0 0x7f2380003d80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2384009ab0 tx=0x7f23800057f0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f23988b5640 1 -- 172.21.15.50:0/2449753478 reap_dead start 2023-10-19T10:43:15.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f239a318640 1 -- 172.21.15.50:0/2449753478 shutdown_connections 2023-10-19T10:43:15.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f239a318640 1 -- 172.21.15.50:0/2449753478 >> 172.21.15.50:0/2449753478 conn(0x7f23940d2370 msgr2=0x7f23940d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:43:15.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f239a318640 1 -- 172.21.15.50:0/2449753478 shutdown_connections 2023-10-19T10:43:15.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.665+0000 7f239a318640 1 -- 172.21.15.50:0/2449753478 wait complete. 2023-10-19T10:43:15.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.667+0000 7f239a318640 1 Processor -- start 2023-10-19T10:43:15.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.667+0000 7f239a318640 1 -- start start 2023-10-19T10:43:15.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f239a318640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23800039a0 0x7f239415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f239a318640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f239415f2f0 0x7f23941f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f239a318640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23941f56e0 0x7f23941f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f239a318640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23941f5b60 con 0x7f23941f56e0 2023-10-19T10:43:15.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f239a318640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239415fb00 con 0x7f239415f2f0 2023-10-19T10:43:15.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f239a318640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2394158550 con 0x7f23800039a0 2023-10-19T10:43:15.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23941f56e0 0x7f23941f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23941f56e0 0x7f23941f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58486/0 (socket says 172.21.15.50:58486) 2023-10-19T10:43:15.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 -- 172.21.15.50:0/1665969029 learned_addr learned my addr 172.21.15.50:0/1665969029 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:43:15.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f2393fff640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23800039a0 0x7f239415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23937fe640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f239415f2f0 0x7f23941f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 -- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23800039a0 msgr2=0x7f239415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23800039a0 0x7f239415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 -- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f239415f2f0 msgr2=0x7f23941f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f239415f2f0 0x7f23941f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 -- 172.21.15.50:0/1665969029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239415f820 con 0x7f23941f56e0 2023-10-19T10:43:15.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.669+0000 7f23988b5640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23941f56e0 0x7f23941f4150 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f23940c93c0 tx=0x7f238c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:43:15.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.670+0000 7f2390ff9640 1 -- 172.21.15.50:0/1665969029 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00eb90 con 0x7f23941f56e0 2023-10-19T10:43:15.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.670+0000 7f2390ff9640 1 -- 172.21.15.50:0/1665969029 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c00ecf0 con 0x7f23941f56e0 2023-10-19T10:43:15.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.670+0000 7f239a318640 1 -- 172.21.15.50:0/1665969029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2394157f00 con 0x7f23941f56e0 2023-10-19T10:43:15.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.670+0000 7f2390ff9640 1 -- 172.21.15.50:0/1665969029 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c014630 con 0x7f23941f56e0 2023-10-19T10:43:15.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.670+0000 7f239a318640 1 -- 172.21.15.50:0/1665969029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23941424a0 con 0x7f23941f56e0 2023-10-19T10:43:15.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.670+0000 7f239a318640 1 -- 172.21.15.50:0/1665969029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f2394066e80 con 0x7f23941f56e0 2023-10-19T10:43:15.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.671+0000 7f2390ff9640 1 -- 172.21.15.50:0/1665969029 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f238c014790 con 0x7f23941f56e0 2023-10-19T10:43:15.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.671+0000 7f2390ff9640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f237406a080 0x7f237406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:43:15.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.672+0000 7f2390ff9640 1 -- 172.21.15.50:0/1665969029 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f238c088480 con 0x7f23941f56e0 2023-10-19T10:43:15.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.672+0000 7f2390ff9640 1 -- 172.21.15.50:0/1665969029 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+446 (secure 0 0 0) 0x7f238c08f070 con 0x7f23941f56e0 2023-10-19T10:43:15.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.672+0000 7f2393fff640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f237406a080 0x7f237406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:43:15.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.672+0000 7f239a318640 1 -- 172.21.15.50:0/1665969029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f2394158690 con 0x7f23941f56e0 2023-10-19T10:43:15.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.672+0000 7f2393fff640 1 --2- 172.21.15.50:0/1665969029 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f237406a080 0x7f237406c540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2384035000 tx=0x7f2384030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:43:15.868 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.866+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d08073990 con 0x7f6d08156510 2023-10-19T10:43:15.869 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.866+0000 7f6d057fa640 1 -- 172.21.15.50:0/1587622172 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f6cf4079180 con 0x7f6d08156510 2023-10-19T10:43:15.870 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:43:15.870 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:43:15.870 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:43:15.871 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6cec06a050 msgr2=0x7f6cec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6cec06a050 0x7f6cec06c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000f60 tx=0x7f6cfc009040 comp rx=0 tx=0).stop 2023-10-19T10:43:15.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 msgr2=0x7f6d0814f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:43:15.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 0x7f6d0814f040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6cf402d080 tx=0x7f6cf40046c0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 shutdown_connections 2023-10-19T10:43:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f6cec06a050 0x7f6cec06c510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d08159750 0x7f6d08150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d08156510 0x7f6d0814f040 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 --2- 172.21.15.50:0/1587622172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d080caf90 0x7f6d080c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:43:15.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 >> 172.21.15.50:0/1587622172 conn(0x7f6d08069e90 msgr2=0x7f6d080cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:43:15.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 shutdown_connections 2023-10-19T10:43:15.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:43:15.868+0000 7f6d0eba6640 1 -- 172.21.15.50:0/1587622172 wait complete. 2023-10-19T10:43:15.882 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:43:16.005 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":275.53904464099998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.52907586685916552,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":277.12800454799998,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.52713704398946104,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:43:16.006 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.5290758668591655 }, "state": "open", "uptime": 275.539044641 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.527137043989461 }, "state": "stale", "uptime": 277.128004548 } ] 2023-10-19T10:43:16.107 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:43:26.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:43:26.998 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:43:57.125 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:43:57.128 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:44:16.120 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:44:16.121 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: '1 client(s) laggy due to laggy OSDs', retrying 2023-10-19T10:44:17.122 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:44:17.126 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:44:17.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.241+0000 7f5db88ab640 1 Processor -- start 2023-10-19T10:44:17.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.241+0000 7f5db88ab640 1 -- start start 2023-10-19T10:44:17.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db88ab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5db415c570 0x7f5db40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db88ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c950 0x7f5db40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db88ab640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40cde60 con 0x7f5db406b760 2023-10-19T10:44:17.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db88ab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db406aa30 con 0x7f5db415c570 2023-10-19T10:44:17.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db88ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db406b2d0 con 0x7f5db415c950 2023-10-19T10:44:17.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db2d9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5db406b760 0x7f5db40d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43764/0 (socket says 172.21.15.50:43764) 2023-10-19T10:44:17.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db2d9d640 1 -- 172.21.15.50:0/3241299298 learned_addr learned my addr 172.21.15.50:0/3241299298 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:44:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db259c640 1 --2- 172.21.15.50:0/3241299298 >> v2:172.21.15.183:3300/0 conn(0x7f5db415c570 0x7f5db40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5da9d9b640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c950 0x7f5db40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db1d9b640 1 -- 172.21.15.50:0/3241299298 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949574060 0 0) 0x7f5db40cde60 con 0x7f5db406b760 2023-10-19T10:44:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.242+0000 7f5db1d9b640 1 -- 172.21.15.50:0/3241299298 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da0003520 con 0x7f5db406b760 2023-10-19T10:44:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.243+0000 7f5db259c640 1 -- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c950 msgr2=0x7f5db40cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:44:17.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.243+0000 7f5db259c640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c950 0x7f5db40cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.243+0000 7f5db259c640 1 -- 172.21.15.50:0/3241299298 >> v1:172.21.15.50:6789/0 conn(0x7f5db406b760 legacy=0x7f5db40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.243+0000 7f5db259c640 1 -- 172.21.15.50:0/3241299298 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40ce090 con 0x7f5db415c570 2023-10-19T10:44:17.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.243+0000 7f5da9d9b640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c950 0x7f5db40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:44:17.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.243+0000 7f5db259c640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c570 0x7f5db40d0e30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009790 tx=0x7f5da402b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46590365e2afd9c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:44:17.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db1d9b640 1 -- 172.21.15.50:0/3241299298 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4031770 con 0x7f5db415c570 2023-10-19T10:44:17.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db1d9b640 1 -- 172.21.15.50:0/3241299298 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40318d0 con 0x7f5db415c570 2023-10-19T10:44:17.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db1d9b640 1 -- 172.21.15.50:0/3241299298 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4031c60 con 0x7f5db415c570 2023-10-19T10:44:17.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 -- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c570 msgr2=0x7f5db40d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c570 0x7f5db40d0e30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009790 tx=0x7f5da402b810 comp rx=0 tx=0).stop 2023-10-19T10:44:17.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 -- 172.21.15.50:0/3241299298 shutdown_connections 2023-10-19T10:44:17.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c950 0x7f5db40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 --2- 172.21.15.50:0/3241299298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c570 0x7f5db40d0e30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 -- 172.21.15.50:0/3241299298 >> 172.21.15.50:0/3241299298 conn(0x7f5db4089ff0 msgr2=0x7f5db409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:44:17.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 -- 172.21.15.50:0/3241299298 shutdown_connections 2023-10-19T10:44:17.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.244+0000 7f5db88ab640 1 -- 172.21.15.50:0/3241299298 wait complete. 2023-10-19T10:44:17.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.246+0000 7f5db88ab640 1 Processor -- start 2023-10-19T10:44:17.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.246+0000 7f5db88ab640 1 -- start start 2023-10-19T10:44:17.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db88ab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 0x7f5db40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db88ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c570 0x7f5db4152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db88ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c950 0x7f5db40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db88ab640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c8bb0 con 0x7f5db406b760 2023-10-19T10:44:17.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db88ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cad50 con 0x7f5db415c950 2023-10-19T10:44:17.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db88ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41533c0 con 0x7f5db415c570 2023-10-19T10:44:17.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 0x7f5db40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 0x7f5db40b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36866/0 (socket says 172.21.15.50:36866) 2023-10-19T10:44:17.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 -- 172.21.15.50:0/1851545557 learned_addr learned my addr 172.21.15.50:0/1851545557 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:44:17.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db2d9d640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c950 0x7f5db40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5da9d9b640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c570 0x7f5db4152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 -- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c570 msgr2=0x7f5db4152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c570 0x7f5db4152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 -- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c950 msgr2=0x7f5db40cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c950 0x7f5db40cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 -- 172.21.15.50:0/1851545557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c009400 con 0x7f5db406b760 2023-10-19T10:44:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5da9d9b640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c570 0x7f5db4152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:44:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.247+0000 7f5db259c640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 0x7f5db40b0d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5da4000c00 tx=0x7f5da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:44:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.248+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4032610 con 0x7f5db406b760 2023-10-19T10:44:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.248+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da4032770 con 0x7f5db406b760 2023-10-19T10:44:17.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.248+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da40093a0 con 0x7f5db406b760 2023-10-19T10:44:17.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.248+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4032b00 con 0x7f5db406b760 2023-10-19T10:44:17.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.248+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db40cafc0 con 0x7f5db406b760 2023-10-19T10:44:17.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.249+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db4073990 con 0x7f5db406b760 2023-10-19T10:44:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.249+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5da4032c60 con 0x7f5db406b760 2023-10-19T10:44:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.250+0000 7f5dab7fe640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9406a050 0x7f5d9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.250+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5da40aa4b0 con 0x7f5db406b760 2023-10-19T10:44:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.250+0000 7f5da9d9b640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9406a050 0x7f5d9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.250+0000 7f5da9d9b640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9406a050 0x7f5d9406c510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5db4154250 tx=0x7f5d9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:44:17.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.258+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5da4075ac0 con 0x7f5db406b760 2023-10-19T10:44:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.269+0000 7fa6ad907640 1 Processor -- start 2023-10-19T10:44:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.269+0000 7fa6ad907640 1 -- start start 2023-10-19T10:44:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6ad907640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa6a81591a0 0x7fa6a80b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6ad907640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a8070b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6ad907640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a80cb6a0 con 0x7fa6a80ca890 2023-10-19T10:44:17.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6ad907640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a80710d0 con 0x7fa6a81591a0 2023-10-19T10:44:17.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6ad907640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80cce20 con 0x7fa6a8070700 2023-10-19T10:44:17.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa6a81591a0 0x7fa6a80b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a67fc640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa6a80ca890 0x7fa6a806e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43768/0 (socket says 172.21.15.50:43768) 2023-10-19T10:44:17.274 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a67fc640 1 -- 172.21.15.50:0/2333292127 learned_addr learned my addr 172.21.15.50:0/2333292127 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:44:17.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a77fe640 1 --2- 172.21.15.50:0/2333292127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a8070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a6ffd640 1 -- 172.21.15.50:0/2333292127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 msgr2=0x7fa6a8070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:44:17.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a6ffd640 1 --2- 172.21.15.50:0/2333292127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a8070b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a6ffd640 1 -- 172.21.15.50:0/2333292127 >> v1:172.21.15.50:6789/0 conn(0x7fa6a80ca890 legacy=0x7fa6a806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.275 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.270+0000 7fa6a6ffd640 1 -- 172.21.15.50:0/2333292127 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8159580 con 0x7fa6a81591a0 2023-10-19T10:44:17.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.271+0000 7fa6a6ffd640 1 --2- 172.21.15.50:0/2333292127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a81591a0 0x7fa6a80b1ef0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa698009790 tx=0x7fa69802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f29917aa7b002b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:44:17.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.271+0000 7fa6a5ffb640 1 -- 172.21.15.50:0/2333292127 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698031770 con 0x7fa6a81591a0 2023-10-19T10:44:17.278 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.271+0000 7fa6a5ffb640 1 -- 172.21.15.50:0/2333292127 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6980318d0 con 0x7fa6a81591a0 2023-10-19T10:44:17.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.271+0000 7fa6a5ffb640 1 -- 172.21.15.50:0/2333292127 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698031c60 con 0x7fa6a81591a0 2023-10-19T10:44:17.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 -- 172.21.15.50:0/2333292127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a81591a0 msgr2=0x7fa6a80b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 --2- 172.21.15.50:0/2333292127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a81591a0 0x7fa6a80b1ef0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa698009790 tx=0x7fa69802b810 comp rx=0 tx=0).stop 2023-10-19T10:44:17.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 -- 172.21.15.50:0/2333292127 shutdown_connections 2023-10-19T10:44:17.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 --2- 172.21.15.50:0/2333292127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a8070b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.279 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 --2- 172.21.15.50:0/2333292127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a81591a0 0x7fa6a80b1ef0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 -- 172.21.15.50:0/2333292127 >> 172.21.15.50:0/2333292127 conn(0x7fa6a80cf020 msgr2=0x7fa6a80cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:44:17.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 -- 172.21.15.50:0/2333292127 shutdown_connections 2023-10-19T10:44:17.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.272+0000 7fa6ad907640 1 -- 172.21.15.50:0/2333292127 wait complete. 2023-10-19T10:44:17.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.273+0000 7fa6ad907640 1 Processor -- start 2023-10-19T10:44:17.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.273+0000 7fa6ad907640 1 -- start start 2023-10-19T10:44:17.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6ad907640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a81f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6ad907640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a80ca890 0x7fa6a8157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6ad907640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a81591a0 0x7fa6a80b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6ad907640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a81f53f0 con 0x7fa6a80ca890 2023-10-19T10:44:17.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6ad907640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8157df0 con 0x7fa6a81591a0 2023-10-19T10:44:17.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6ad907640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a81f4d50 con 0x7fa6a8070700 2023-10-19T10:44:17.281 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6a67fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a80ca890 0x7fa6a8157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6a67fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a80ca890 0x7fa6a8157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36894/0 (socket says 172.21.15.50:36894) 2023-10-19T10:44:17.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6a67fc640 1 -- 172.21.15.50:0/3953337377 learned_addr learned my addr 172.21.15.50:0/3953337377 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:44:17.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6a6ffd640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a81f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.274+0000 7fa6a77fe640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a81591a0 0x7fa6a80b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.282 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6a67fc640 1 -- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 msgr2=0x7fa6a81f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6a67fc640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6a8070700 0x7fa6a81f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6a67fc640 1 -- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a81591a0 msgr2=0x7fa6a80b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6a67fc640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a81591a0 0x7fa6a80b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6a67fc640 1 -- 172.21.15.50:0/3953337377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980093a0 con 0x7fa6a80ca890 2023-10-19T10:44:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6a67fc640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6a80ca890 0x7fa6a8157210 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c400 tx=0x7fa69c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:44:17.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa68ffff640 1 -- 172.21.15.50:0/3953337377 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c005ce0 con 0x7fa6a80ca890 2023-10-19T10:44:17.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa68ffff640 1 -- 172.21.15.50:0/3953337377 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c005e40 con 0x7fa6a80ca890 2023-10-19T10:44:17.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6ad907640 1 -- 172.21.15.50:0/3953337377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a81de7b0 con 0x7fa6a80ca890 2023-10-19T10:44:17.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa68ffff640 1 -- 172.21.15.50:0/3953337377 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0072d0 con 0x7fa6a80ca890 2023-10-19T10:44:17.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.275+0000 7fa6ad907640 1 -- 172.21.15.50:0/3953337377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a81f5010 con 0x7fa6a80ca890 2023-10-19T10:44:17.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.276+0000 7fa6ad907640 1 -- 172.21.15.50:0/3953337377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa6a81f5530 con 0x7fa6a80ca890 2023-10-19T10:44:17.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.277+0000 7fa68ffff640 1 -- 172.21.15.50:0/3953337377 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa69c0074f0 con 0x7fa6a80ca890 2023-10-19T10:44:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.277+0000 7fa68ffff640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa69406a050 0x7fa69406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:44:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.277+0000 7fa68ffff640 1 -- 172.21.15.50:0/3953337377 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa69c087ae0 con 0x7fa6a80ca890 2023-10-19T10:44:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.277+0000 7fa68ffff640 1 -- 172.21.15.50:0/3953337377 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+446 (secure 0 0 0) 0x7fa6a81f5530 con 0x7fa6a80ca890 2023-10-19T10:44:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.277+0000 7fa6a6ffd640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa69406a050 0x7fa69406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:44:17.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.277+0000 7fa6ad907640 1 -- 172.21.15.50:0/3953337377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fa6a80cb6a0 con 0x7fa6a80ca890 2023-10-19T10:44:17.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.278+0000 7fa6a6ffd640 1 --2- 172.21.15.50:0/3953337377 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa69406a050 0x7fa69406c510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa698036000 tx=0x7fa69803d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:44:17.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.473+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5db40d3480 con 0x7f5db406b760 2023-10-19T10:44:17.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.474+0000 7f5dab7fe640 1 -- 172.21.15.50:0/1851545557 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f5da40790e0 con 0x7f5db406b760 2023-10-19T10:44:17.477 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:44:17.477 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:44:17.478 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:44:17.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9406a050 msgr2=0x7f5d9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9406a050 0x7f5d9406c510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5db4154250 tx=0x7f5d9c008040 comp rx=0 tx=0).stop 2023-10-19T10:44:17.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 msgr2=0x7f5db40b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:44:17.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 0x7f5db40b0d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5da4000c00 tx=0x7f5da40046c0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 shutdown_connections 2023-10-19T10:44:17.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d9406a050 0x7f5d9406c510 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db415c950 0x7f5db40cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5db415c570 0x7f5db4152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 --2- 172.21.15.50:0/1851545557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5db406b760 0x7f5db40b0d10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:44:17.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 >> 172.21.15.50:0/1851545557 conn(0x7f5db4089ff0 msgr2=0x7f5db415b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:44:17.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 shutdown_connections 2023-10-19T10:44:17.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:44:17.476+0000 7f5db88ab640 1 -- 172.21.15.50:0/1851545557 wait complete. 2023-10-19T10:44:17.489 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:44:17.611 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":337.14548716500002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.4588809144343628,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":338.73544704699998,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.45719826787207696,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:44:17.612 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.4588809144343628 }, "state": "open", "uptime": 337.145487165 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.45719826787207696 }, "state": "stale", "uptime": 338.735447047 } ] 2023-10-19T10:44:17.713 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:44:27.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:44:27.265 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:44:57.401 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:44:57.404 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:45:17.726 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:45:17.727 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: '1 client(s) laggy due to laggy OSDs', retrying 2023-10-19T10:45:18.728 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:45:18.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:45:18.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.847+0000 7faae4fa0640 1 Processor -- start 2023-10-19T10:45:18.850 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.847+0000 7faae4fa0640 1 -- start start 2023-10-19T10:45:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faae4fa0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faae00b23c0 0x7faae00ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faae4fa0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae00caf80 con 0x7faae00b1f30 2023-10-19T10:45:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faae4fa0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae00d02e0 con 0x7faae015a370 2023-10-19T10:45:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faae4fa0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae00b2850 con 0x7faae00b23c0 2023-10-19T10:45:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faaded9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faae00b23c0 0x7faae00ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faad7fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7faae00b1f30 0x7faae00ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57978/0 (socket says 172.21.15.50:57978) 2023-10-19T10:45:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.848+0000 7faad7fff640 1 -- 172.21.15.50:0/3581572408 learned_addr learned my addr 172.21.15.50:0/3581572408 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:45:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.849+0000 7faaded9d640 1 -- 172.21.15.50:0/3581572408 >> v1:172.21.15.183:6789/0 conn(0x7faae015a370 legacy=0x7faae006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:45:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.849+0000 7faaded9d640 1 -- 172.21.15.50:0/3581572408 >> v1:172.21.15.50:6789/0 conn(0x7faae00b1f30 legacy=0x7faae00ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.849+0000 7faaded9d640 1 -- 172.21.15.50:0/3581572408 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae015a750 con 0x7faae00b23c0 2023-10-19T10:45:18.854 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.849+0000 7faaddd9b640 1 -- 172.21.15.50:0/3581572408 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112780306 0 0) 0x7faae00caf80 con 0x7faae00b1f30 2023-10-19T10:45:18.856 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faaded9d640 1 --2- 172.21.15.50:0/3581572408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae00b23c0 0x7faae00ce790 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faad8009d90 tx=0x7faad802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65109ec2380cd9fd server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:45:18.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faaddd9b640 1 -- 172.21.15.50:0/3581572408 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad802cc60 con 0x7faae00b23c0 2023-10-19T10:45:18.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faaddd9b640 1 -- 172.21.15.50:0/3581572408 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad802cdc0 con 0x7faae00b23c0 2023-10-19T10:45:18.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faaddd9b640 1 -- 172.21.15.50:0/3581572408 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8034660 con 0x7faae00b23c0 2023-10-19T10:45:18.857 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 -- 172.21.15.50:0/3581572408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae00b23c0 msgr2=0x7faae00ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 --2- 172.21.15.50:0/3581572408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae00b23c0 0x7faae00ce790 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faad8009d90 tx=0x7faad802beb0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 -- 172.21.15.50:0/3581572408 shutdown_connections 2023-10-19T10:45:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 --2- 172.21.15.50:0/3581572408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae00b23c0 0x7faae00ce790 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 -- 172.21.15.50:0/3581572408 >> 172.21.15.50:0/3581572408 conn(0x7faae00ccf30 msgr2=0x7faae00cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:45:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 -- 172.21.15.50:0/3581572408 shutdown_connections 2023-10-19T10:45:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.850+0000 7faae4fa0640 1 -- 172.21.15.50:0/3581572408 wait complete. 2023-10-19T10:45:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.852+0000 7faae4fa0640 1 Processor -- start 2023-10-19T10:45:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.852+0000 7faae4fa0640 1 -- start start 2023-10-19T10:45:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faae00b1f30 0x7faae00c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00b23c0 0x7faae0154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 0x7faae015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae00c8640 con 0x7faae015a370 2023-10-19T10:45:18.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae0155420 con 0x7faae00b23c0 2023-10-19T10:45:18.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae015ce80 con 0x7faae00b1f30 2023-10-19T10:45:18.860 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 0x7faae015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 0x7faae015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57036/0 (socket says 172.21.15.50:57036) 2023-10-19T10:45:18.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 -- 172.21.15.50:0/1574548962 learned_addr learned my addr 172.21.15.50:0/1574548962 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:45:18.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faad7fff640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00b23c0 0x7faae0154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faade59c640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faae00b1f30 0x7faae00c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.861 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 -- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faae00b1f30 msgr2=0x7faae00c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faae00b1f30 0x7faae00c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 -- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00b23c0 msgr2=0x7faae0154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00b23c0 0x7faae0154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 -- 172.21.15.50:0/1574548962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad8009a70 con 0x7faae015a370 2023-10-19T10:45:18.862 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faade59c640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faae00b1f30 0x7faae00c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:45:18.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faaded9d640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 0x7faae015c430 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faad8009d90 tx=0x7faad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:45:18.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad803e610 con 0x7faae015a370 2023-10-19T10:45:18.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad803e790 con 0x7faae015a370 2023-10-19T10:45:18.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae00c4070 con 0x7faae015a370 2023-10-19T10:45:18.863 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.853+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad803eb40 con 0x7faae015a370 2023-10-19T10:45:18.864 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.854+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae00c4630 con 0x7faae015a370 2023-10-19T10:45:18.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.855+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae00c3ed0 con 0x7faae015a370 2023-10-19T10:45:18.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.855+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faad803b050 con 0x7faae015a370 2023-10-19T10:45:18.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.855+0000 7faad6ffd640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faac006a050 0x7faac006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.856+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7faad80ab0f0 con 0x7faae015a370 2023-10-19T10:45:18.865 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.856+0000 7faade59c640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faac006a050 0x7faac006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.866 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.856+0000 7faade59c640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faac006a050 0x7faac006c510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faad0003eb0 tx=0x7faad0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:45:18.867 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.864+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7faae00c3ed0 con 0x7faae015a370 2023-10-19T10:45:18.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.875+0000 7f5265bb2640 1 Processor -- start 2023-10-19T10:45:18.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.875+0000 7f5265bb2640 1 -- start start 2023-10-19T10:45:18.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f5265bb2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f526015c680 0x7f52600d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f5265bb2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600cd960 0x7f52600d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f5265bb2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600cde90 con 0x7f52600d0e70 2023-10-19T10:45:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f5265bb2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52600d1300 con 0x7f526015c680 2023-10-19T10:45:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f5265bb2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52600ce170 con 0x7f52600cd960 2023-10-19T10:45:18.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f525f7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f526015c680 0x7f52600d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f525effd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600cd960 0x7f52600d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f525ffff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f52600d0e70 0x7f52600ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:53456/0 (socket says 172.21.15.50:53456) 2023-10-19T10:45:18.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.876+0000 7f525ffff640 1 -- 172.21.15.50:0/621817950 learned_addr learned my addr 172.21.15.50:0/621817950 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:45:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.877+0000 7f525f7fe640 1 -- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600cd960 msgr2=0x7f52600d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:45:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.877+0000 7f525f7fe640 1 --2- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600cd960 0x7f52600d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.877+0000 7f525f7fe640 1 -- 172.21.15.50:0/621817950 >> v1:172.21.15.183:6789/0 conn(0x7f52600d0e70 legacy=0x7f52600ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.877+0000 7f525f7fe640 1 -- 172.21.15.50:0/621817950 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600b1c80 con 0x7f526015c680 2023-10-19T10:45:18.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.877+0000 7f525e7fc640 1 -- 172.21.15.50:0/621817950 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008808073 0 0) 0x7f52600cde90 con 0x7f52600d0e70 2023-10-19T10:45:18.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f525f7fe640 1 --2- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 0x7f52600d0090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5250009790 tx=0x7f525002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9a5c2f1fce6a15 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:45:18.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f525e7fc640 1 -- 172.21.15.50:0/621817950 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250031890 con 0x7f526015c680 2023-10-19T10:45:18.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f525e7fc640 1 -- 172.21.15.50:0/621817950 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52500319f0 con 0x7f526015c680 2023-10-19T10:45:18.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f525e7fc640 1 -- 172.21.15.50:0/621817950 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250031d80 con 0x7f526015c680 2023-10-19T10:45:18.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 -- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 msgr2=0x7f52600d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 --2- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 0x7f52600d0090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5250009790 tx=0x7f525002b930 comp rx=0 tx=0).stop 2023-10-19T10:45:18.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 -- 172.21.15.50:0/621817950 shutdown_connections 2023-10-19T10:45:18.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 --2- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600cd960 0x7f52600d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 --2- 172.21.15.50:0/621817950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 0x7f52600d0090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 -- 172.21.15.50:0/621817950 >> 172.21.15.50:0/621817950 conn(0x7f52600d2370 msgr2=0x7f52600d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:45:18.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 -- 172.21.15.50:0/621817950 shutdown_connections 2023-10-19T10:45:18.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.878+0000 7f5265bb2640 1 -- 172.21.15.50:0/621817950 wait complete. 2023-10-19T10:45:18.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 Processor -- start 2023-10-19T10:45:18.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 -- start start 2023-10-19T10:45:18.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52600cd960 0x7f526014a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600d0e70 0x7f526015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 0x7f526015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526015d7d0 con 0x7f52600cd960 2023-10-19T10:45:18.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526015dab0 con 0x7f526015c680 2023-10-19T10:45:18.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.880+0000 7f5265bb2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526015dd90 con 0x7f52600d0e70 2023-10-19T10:45:18.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52600cd960 0x7f526014a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52600cd960 0x7f526014a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57040/0 (socket says 172.21.15.50:57040) 2023-10-19T10:45:18.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 -- 172.21.15.50:0/1981640217 learned_addr learned my addr 172.21.15.50:0/1981640217 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:45:18.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525ffff640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 0x7f526015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525effd640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600d0e70 0x7f526015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 -- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600d0e70 msgr2=0x7f526015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f52600d0e70 0x7f526015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 -- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 msgr2=0x7f526015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:18.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526015c680 0x7f526015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:18.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 -- 172.21.15.50:0/1981640217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500093a0 con 0x7f52600cd960 2023-10-19T10:45:18.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f525f7fe640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52600cd960 0x7f526014a890 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5250034040 tx=0x7f52500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:45:18.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250005e60 con 0x7f52600cd960 2023-10-19T10:45:18.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525003a850 con 0x7f52600cd960 2023-10-19T10:45:18.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f5265bb2640 1 -- 172.21.15.50:0/1981640217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526015e070 con 0x7f52600cd960 2023-10-19T10:45:18.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525003ac00 con 0x7f52600cd960 2023-10-19T10:45:18.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.881+0000 7f5265bb2640 1 -- 172.21.15.50:0/1981640217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52600c7080 con 0x7f52600cd960 2023-10-19T10:45:18.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.882+0000 7f5265bb2640 1 -- 172.21.15.50:0/1981640217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5260066e80 con 0x7f52600cd960 2023-10-19T10:45:18.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.883+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5250006030 con 0x7f52600cd960 2023-10-19T10:45:18.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.883+0000 7f5247fff640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f524c06a050 0x7f524c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:45:18.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.883+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f52500ab400 con 0x7f52600cd960 2023-10-19T10:45:18.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.883+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+446 (secure 0 0 0) 0x7f52500ab790 con 0x7f52600cd960 2023-10-19T10:45:18.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.883+0000 7f525effd640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f524c06a050 0x7f524c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:45:18.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.884+0000 7f5265bb2640 1 -- 172.21.15.50:0/1981640217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f52600b7ea0 con 0x7f52600cd960 2023-10-19T10:45:18.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:18.884+0000 7f525effd640 1 --2- 172.21.15.50:0/1981640217 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f524c06a050 0x7f524c06c510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5260159c20 tx=0x7f5254009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:45:19.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.075+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faae00c3ed0 con 0x7faae015a370 2023-10-19T10:45:19.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.075+0000 7faad6ffd640 1 -- 172.21.15.50:0/1574548962 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4881 (secure 0 0 0) 0x7faae00c3ed0 con 0x7faae015a370 2023-10-19T10:45:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:45:19.079 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:38:30.587775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:45:19.079 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 15 2023-10-19T10:45:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faac006a050 msgr2=0x7faac006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faac006a050 0x7faac006c510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faad0003eb0 tx=0x7faad0009040 comp rx=0 tx=0).stop 2023-10-19T10:45:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 msgr2=0x7faae015c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:45:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 0x7faae015c430 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faad8009d90 tx=0x7faad8004a40 comp rx=0 tx=0).stop 2023-10-19T10:45:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 shutdown_connections 2023-10-19T10:45:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7faac006a050 0x7faac006c510 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faae015a370 0x7faae015c430 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00b23c0 0x7faae0154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 --2- 172.21.15.50:0/1574548962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faae00b1f30 0x7faae00c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:45:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 >> 172.21.15.50:0/1574548962 conn(0x7faae00ccf30 msgr2=0x7faae00cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:45:19.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 shutdown_connections 2023-10-19T10:45:19.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:19.077+0000 7faae4fa0640 1 -- 172.21.15.50:0/1574548962 wait complete. 2023-10-19T10:45:19.091 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:45:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":398.74792635,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.39800273373148798,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":400.336886257,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.39654423439198028,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:45:19.215 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.398002733731488 }, "state": "open", "uptime": 398.74792635 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.3965442343919803 }, "state": "stale", "uptime": 400.336886257 } ] 2023-10-19T10:45:19.316 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:45:27.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:45:27.466 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:45:46.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:45:46.198+0000 7f5247fff640 1 -- 172.21.15.50:0/1981640217 <== mon.0 v2:172.21.15.50:3300/0 7 ==== log(1 entries from seq 481 at 2023-10-19T10:45:45.408846+0000) v1 ==== 246+0+0 (secure 0 0 0) 0x7f525007e360 con 0x7f52600cd960 2023-10-19T10:45:46.203 INFO:teuthology.orchestra.run.smithi050.stdout: cluster: 2023-10-19T10:45:46.203 INFO:teuthology.orchestra.run.smithi050.stdout: id: 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:45:46.203 INFO:teuthology.orchestra.run.smithi050.stdout: health: HEALTH_OK 2023-10-19T10:45:46.203 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:45:46.204 INFO:teuthology.orchestra.run.smithi050.stdout: services: 2023-10-19T10:45:46.204 INFO:teuthology.orchestra.run.smithi050.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2023-10-19T10:45:46.204 INFO:teuthology.orchestra.run.smithi050.stdout: mgr: x(active, since 8m), standbys: y 2023-10-19T10:45:46.204 INFO:teuthology.orchestra.run.smithi050.stdout: mds: 1/1 daemons up, 3 standby 2023-10-19T10:45:46.204 INFO:teuthology.orchestra.run.smithi050.stdout: osd: 8 osds: 8 up (since 4m), 8 in (since 8m) 2023-10-19T10:45:46.205 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:45:46.205 INFO:teuthology.orchestra.run.smithi050.stdout: data: 2023-10-19T10:45:46.205 INFO:teuthology.orchestra.run.smithi050.stdout: volumes: 1/1 healthy 2023-10-19T10:45:46.205 INFO:teuthology.orchestra.run.smithi050.stdout: pools: 4 pools, 105 pgs 2023-10-19T10:45:46.205 INFO:teuthology.orchestra.run.smithi050.stdout: objects: 24 objects, 581 KiB 2023-10-19T10:45:46.206 INFO:teuthology.orchestra.run.smithi050.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2023-10-19T10:45:46.206 INFO:teuthology.orchestra.run.smithi050.stdout: pgs: 105 active+clean 2023-10-19T10:45:46.206 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:45:46.206 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:45:46.207 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:45:45.408846+0000 mon.a [WRN] Health check failed: 1 client(s) laggy due to laggy OSDs (MDS_CLIENTS_LAGGY) 2023-10-19T10:45:57.560 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:45:57.563 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:46:19.330 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mon mon_osd_laggy_weight 2023-10-19T10:46:19.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.486+0000 7f922f2f9640 1 Processor -- start 2023-10-19T10:46:19.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922f2f9640 1 -- start start 2023-10-19T10:46:19.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922f2f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f922815c620 0x7f92280b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922f2f9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f92280cfb00 0x7f92280d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922f2f9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92280cda30 con 0x7f92280b82f0 2023-10-19T10:46:19.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922f2f9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280ccc10 con 0x7f92280cfb00 2023-10-19T10:46:19.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922f2f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9228161110 con 0x7f922815c620 2023-10-19T10:46:19.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922d896640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f92280cfb00 0x7f92280d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922d896640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f92280cfb00 0x7f92280d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47874/0 (socket says 172.21.15.50:47874) 2023-10-19T10:46:19.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.487+0000 7f922d896640 1 -- 172.21.15.50:0/1205160744 learned_addr learned my addr 172.21.15.50:0/1205160744 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.488+0000 7f922d095640 1 --2- 172.21.15.50:0/1205160744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f922815c620 0x7f92280b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.488+0000 7f922d896640 1 -- 172.21.15.50:0/1205160744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f922815c620 msgr2=0x7f92280b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.488+0000 7f922d896640 1 --2- 172.21.15.50:0/1205160744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f922815c620 0x7f92280b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.488+0000 7f922d896640 1 -- 172.21.15.50:0/1205160744 >> v1:172.21.15.183:6789/0 conn(0x7f92280b82f0 legacy=0x7f92280cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.488+0000 7f922d896640 1 -- 172.21.15.50:0/1205160744 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280cce20 con 0x7f92280cfb00 2023-10-19T10:46:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922d896640 1 --2- 172.21.15.50:0/1205160744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92280cfb00 0x7f92280d0b00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f922402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfea2fb0701c8047 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f921ffff640 1 -- 172.21.15.50:0/1205160744 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922402cb00 con 0x7f92280cfb00 2023-10-19T10:46:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f921ffff640 1 -- 172.21.15.50:0/1205160744 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922402cc60 con 0x7f92280cfb00 2023-10-19T10:46:19.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f921ffff640 1 -- 172.21.15.50:0/1205160744 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240343f0 con 0x7f92280cfb00 2023-10-19T10:46:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 -- 172.21.15.50:0/1205160744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92280cfb00 msgr2=0x7f92280d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 --2- 172.21.15.50:0/1205160744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92280cfb00 0x7f92280d0b00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f922402bf60 comp rx=0 tx=0).stop 2023-10-19T10:46:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 -- 172.21.15.50:0/1205160744 shutdown_connections 2023-10-19T10:46:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 --2- 172.21.15.50:0/1205160744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92280cfb00 0x7f92280d0b00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 --2- 172.21.15.50:0/1205160744 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f922815c620 0x7f92280b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 -- 172.21.15.50:0/1205160744 >> 172.21.15.50:0/1205160744 conn(0x7f9228071110 msgr2=0x7f92280714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:19.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 -- 172.21.15.50:0/1205160744 shutdown_connections 2023-10-19T10:46:19.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.489+0000 7f922f2f9640 1 -- 172.21.15.50:0/1205160744 wait complete. 2023-10-19T10:46:19.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 Processor -- start 2023-10-19T10:46:19.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 -- start start 2023-10-19T10:46:19.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92280b82f0 0x7f92280cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92280cfb00 0x7f92280b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 0x7f92280ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922815dfc0 con 0x7f922815c620 2023-10-19T10:46:19.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922815d410 con 0x7f92280b82f0 2023-10-19T10:46:19.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922f2f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b0cb0 con 0x7f92280cfb00 2023-10-19T10:46:19.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.491+0000 7f922d896640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 0x7f92280ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 0x7f92280ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47880/0 (socket says 172.21.15.50:47880) 2023-10-19T10:46:19.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 -- 172.21.15.50:0/2003271800 learned_addr learned my addr 172.21.15.50:0/2003271800 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:19.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922c894640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92280cfb00 0x7f92280b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d095640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92280b82f0 0x7f92280cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 -- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92280cfb00 msgr2=0x7f92280b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92280cfb00 0x7f92280b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 -- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92280b82f0 msgr2=0x7f92280cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92280b82f0 0x7f92280cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 -- 172.21.15.50:0/2003271800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9224009b30 con 0x7f922815c620 2023-10-19T10:46:19.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d095640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92280b82f0 0x7f92280cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:46:19.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922c894640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92280cfb00 0x7f92280b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:46:19.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922d896640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 0x7f92280ca010 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f9224004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:19.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224009510 con 0x7f922815c620 2023-10-19T10:46:19.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922403e450 con 0x7f922815c620 2023-10-19T10:46:19.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922403e800 con 0x7f922815c620 2023-10-19T10:46:19.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.492+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922815d620 con 0x7f922815c620 2023-10-19T10:46:19.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.493+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922815dbe0 con 0x7f922815c620 2023-10-19T10:46:19.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.494+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922815e100 con 0x7f922815c620 2023-10-19T10:46:19.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.494+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f922403e960 con 0x7f922815c620 2023-10-19T10:46:19.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.494+0000 7f921dffb640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f920c06a050 0x7f920c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.495+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f92240ab4f0 con 0x7f922815c620 2023-10-19T10:46:19.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.495+0000 7f922d095640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f920c06a050 0x7f920c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.495+0000 7f922d095640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f920c06a050 0x7f920c06c510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9220006fd0 tx=0x7f9220008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:19.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.499+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f922815e100 con 0x7f922815c620 2023-10-19T10:46:19.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.675+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mon", "key": "mon_osd_laggy_weight"} v 0) v1 -- 0x7f922815e100 con 0x7f922815c620 2023-10-19T10:46:19.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.675+0000 7f921dffb640 1 -- 172.21.15.50:0/2003271800 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mon", "key": "mon_osd_laggy_weight"}]=0 v11) v1 ==== 103+0+9 (secure 0 0 0) 0x7f922815e100 con 0x7f922815c620 2023-10-19T10:46:19.678 INFO:teuthology.orchestra.run.smithi050.stdout:0.300000 2023-10-19T10:46:19.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f920c06a050 msgr2=0x7f920c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f920c06a050 0x7f920c06c510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9220006fd0 tx=0x7f9220008040 comp rx=0 tx=0).stop 2023-10-19T10:46:19.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 msgr2=0x7f92280ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 0x7f92280ca010 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f9224004a40 comp rx=0 tx=0).stop 2023-10-19T10:46:19.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 shutdown_connections 2023-10-19T10:46:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f920c06a050 0x7f920c06c510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922815c620 0x7f92280ca010 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92280cfb00 0x7f92280b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 --2- 172.21.15.50:0/2003271800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92280b82f0 0x7f92280cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 >> 172.21.15.50:0/2003271800 conn(0x7f9228071110 msgr2=0x7f92280b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 shutdown_connections 2023-10-19T10:46:19.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.677+0000 7f922f2f9640 1 -- 172.21.15.50:0/2003271800 wait complete. 2023-10-19T10:46:19.690 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_osd_laggy_weight 1 2023-10-19T10:46:19.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.796+0000 7f7bbb53e640 1 Processor -- start 2023-10-19T10:46:19.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.797+0000 7f7bbb53e640 1 -- start start 2023-10-19T10:46:19.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.797+0000 7f7bbb53e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7bb40cd5a0 0x7f7bb415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.797+0000 7f7bbb53e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7bb40d1010 0x7f7bb415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.797+0000 7f7bbb53e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb40cec80 con 0x7f7bb40ce8a0 2023-10-19T10:46:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.797+0000 7f7bbb53e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40d0b80 con 0x7f7bb40d1010 2023-10-19T10:46:19.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.797+0000 7f7bbb53e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb415dae0 con 0x7f7bb40cd5a0 2023-10-19T10:46:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7bb40d1010 0x7f7bb415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7bb40d1010 0x7f7bb415d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47892/0 (socket says 172.21.15.50:47892) 2023-10-19T10:46:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 -- 172.21.15.50:0/1955576728 learned_addr learned my addr 172.21.15.50:0/1955576728 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb8ad9640 1 --2- 172.21.15.50:0/1955576728 >> v2:172.21.15.183:3300/0 conn(0x7f7bb40cd5a0 0x7f7bb415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 -- 172.21.15.50:0/1955576728 >> v2:172.21.15.183:3300/0 conn(0x7f7bb40cd5a0 msgr2=0x7f7bb415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 --2- 172.21.15.50:0/1955576728 >> v2:172.21.15.183:3300/0 conn(0x7f7bb40cd5a0 0x7f7bb415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 -- 172.21.15.50:0/1955576728 >> v1:172.21.15.183:6789/0 conn(0x7f7bb40ce8a0 legacy=0x7f7bb40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.798+0000 7f7bb9adb640 1 -- 172.21.15.50:0/1955576728 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb415dd90 con 0x7f7bb40d1010 2023-10-19T10:46:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bb9adb640 1 --2- 172.21.15.50:0/1955576728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb415d5a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40c3d30 tx=0x7f7bac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e3a7cc14ad92a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:19.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7babfff640 1 -- 172.21.15.50:0/1955576728 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac02ca80 con 0x7f7bb40d1010 2023-10-19T10:46:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7babfff640 1 -- 172.21.15.50:0/1955576728 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac02cbe0 con 0x7f7bb40d1010 2023-10-19T10:46:19.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7babfff640 1 -- 172.21.15.50:0/1955576728 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0343f0 con 0x7f7bb40d1010 2023-10-19T10:46:19.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 -- 172.21.15.50:0/1955576728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 msgr2=0x7f7bb415d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/1955576728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb415d5a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40c3d30 tx=0x7f7bac02bee0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 -- 172.21.15.50:0/1955576728 shutdown_connections 2023-10-19T10:46:19.811 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/1955576728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb415d5a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/1955576728 >> v2:172.21.15.183:3300/0 conn(0x7f7bb40cd5a0 0x7f7bb415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 -- 172.21.15.50:0/1955576728 >> 172.21.15.50:0/1955576728 conn(0x7f7bb4069800 msgr2=0x7f7bb4069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:19.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 -- 172.21.15.50:0/1955576728 shutdown_connections 2023-10-19T10:46:19.812 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.799+0000 7f7bbb53e640 1 -- 172.21.15.50:0/1955576728 wait complete. 2023-10-19T10:46:19.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 Processor -- start 2023-10-19T10:46:19.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 -- start start 2023-10-19T10:46:19.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bb40cd5a0 0x7f7bb40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bb40ce8a0 0x7f7bb40c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb40b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41f6450 con 0x7f7bb40d1010 2023-10-19T10:46:19.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40ca4f0 con 0x7f7bb40ce8a0 2023-10-19T10:46:19.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bbb53e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40c8620 con 0x7f7bb40cd5a0 2023-10-19T10:46:19.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bb9adb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb40b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bb9adb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb40b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47900/0 (socket says 172.21.15.50:47900) 2023-10-19T10:46:19.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.801+0000 7f7bb9adb640 1 -- 172.21.15.50:0/796142828 learned_addr learned my addr 172.21.15.50:0/796142828 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:19.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb8ad9640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bb40ce8a0 0x7f7bb40c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb92da640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bb40cd5a0 0x7f7bb40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb9adb640 1 -- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bb40cd5a0 msgr2=0x7f7bb40c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb9adb640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bb40cd5a0 0x7f7bb40c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb9adb640 1 -- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bb40ce8a0 msgr2=0x7f7bb40c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb9adb640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bb40ce8a0 0x7f7bb40c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb9adb640 1 -- 172.21.15.50:0/796142828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac009b30 con 0x7f7bb40d1010 2023-10-19T10:46:19.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bb9adb640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb40b5660 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7bac036040 tx=0x7f7bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:19.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac009510 con 0x7f7bb40d1010 2023-10-19T10:46:19.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac03e450 con 0x7f7bb40d1010 2023-10-19T10:46:19.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac03e800 con 0x7f7bb40d1010 2023-10-19T10:46:19.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb41f5690 con 0x7f7bb40d1010 2023-10-19T10:46:19.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.802+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb41f5c50 con 0x7f7bb40d1010 2023-10-19T10:46:19.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.803+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb41f6170 con 0x7f7bb40d1010 2023-10-19T10:46:19.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.804+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7bac02cd50 con 0x7f7bb40d1010 2023-10-19T10:46:19.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.804+0000 7f7ba9ffb640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7b9806a050 0x7f7b9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:19.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.804+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7bac0aaf80 con 0x7f7bb40d1010 2023-10-19T10:46:19.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.804+0000 7f7bb92da640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7b9806a050 0x7f7b9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:19.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.805+0000 7f7bb92da640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7b9806a050 0x7f7b9806c510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0000f60 tx=0x7f7bb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:19.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.809+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7bac0dd940 con 0x7f7bb40d1010 2023-10-19T10:46:19.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.984+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_osd_laggy_weight}] v 0) v1 -- 0x7f7bb40c7280 con 0x7f7bb40d1010 2023-10-19T10:46:19.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.988+0000 7f7ba9ffb640 1 -- 172.21.15.50:0/796142828 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_osd_laggy_weight}]=0 v12)=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7bb41f6170 con 0x7f7bb40d1010 2023-10-19T10:46:19.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7b9806a050 msgr2=0x7f7b9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7b9806a050 0x7f7b9806c510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0000f60 tx=0x7f7bb0009040 comp rx=0 tx=0).stop 2023-10-19T10:46:19.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 msgr2=0x7f7bb40b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:19.993 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb40b5660 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7bac036040 tx=0x7f7bac004a40 comp rx=0 tx=0).stop 2023-10-19T10:46:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 shutdown_connections 2023-10-19T10:46:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7b9806a050 0x7f7b9806c510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bb40d1010 0x7f7bb40b5660 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.994 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bb40ce8a0 0x7f7bb40c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 --2- 172.21.15.50:0/796142828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bb40cd5a0 0x7f7bb40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:19.995 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 >> 172.21.15.50:0/796142828 conn(0x7f7bb4069800 msgr2=0x7f7bb4156900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:19.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 shutdown_connections 2023-10-19T10:46:19.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:19.990+0000 7f7bbb53e640 1 -- 172.21.15.50:0/796142828 wait complete. 2023-10-19T10:46:20.003 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-19T10:46:20.003 INFO:tasks.ceph.osd.2:Stopping old one... 2023-10-19T10:46:20.004 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-19T10:46:20.004 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:46:20.039 INFO:tasks.ceph.osd.2:Stopped 2023-10-19T10:46:20.039 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-19T10:46:20.081 INFO:tasks.ceph.osd.2:Started 2023-10-19T10:46:20.082 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-19T10:46:20.189 INFO:teuthology.orchestra.run.smithi050.stderr:no valid command found; 10 closest matches: 2023-10-19T10:46:20.189 INFO:teuthology.orchestra.run.smithi050.stderr:0 2023-10-19T10:46:20.189 INFO:teuthology.orchestra.run.smithi050.stderr:1 2023-10-19T10:46:20.189 INFO:teuthology.orchestra.run.smithi050.stderr:2 2023-10-19T10:46:20.189 INFO:teuthology.orchestra.run.smithi050.stderr:abort 2023-10-19T10:46:20.190 INFO:teuthology.orchestra.run.smithi050.stderr:assert 2023-10-19T10:46:20.190 INFO:teuthology.orchestra.run.smithi050.stderr:config diff 2023-10-19T10:46:20.190 INFO:teuthology.orchestra.run.smithi050.stderr:config diff get 2023-10-19T10:46:20.190 INFO:teuthology.orchestra.run.smithi050.stderr:config get 2023-10-19T10:46:20.191 INFO:teuthology.orchestra.run.smithi050.stderr:config help [] 2023-10-19T10:46:20.191 INFO:teuthology.orchestra.run.smithi050.stderr:config set ... 2023-10-19T10:46:20.191 INFO:teuthology.orchestra.run.smithi050.stderr:admin_socket: invalid command 2023-10-19T10:46:20.192 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-19T10:46:20.192 INFO:tasks.cephfs.filesystem.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2023-10-19T10:46:20.677 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:46:20.674+0000 7fa150acf500 -1 Falling back to public interface 2023-10-19T10:46:24.161 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:46:24.158+0000 7fa150acf500 -1 osd.2 50 log_to_monitors true 2023-10-19T10:46:24.325 INFO:tasks.ceph.osd.2.smithi050.stderr:2023-10-19T10:46:24.322+0000 7fa14487c640 -1 osd.2 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-19T10:46:25.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-19T10:46:25.322 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:46:25.322 INFO:teuthology.orchestra.run.smithi050.stdout: "ops": [], 2023-10-19T10:46:25.322 INFO:teuthology.orchestra.run.smithi050.stdout: "num_ops": 0 2023-10-19T10:46:25.323 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:46:25.334 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_osd_laggy_weight 0.300000 2023-10-19T10:46:25.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.445+0000 7f691381c640 1 Processor -- start 2023-10-19T10:46:25.448 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.445+0000 7f691381c640 1 -- start start 2023-10-19T10:46:25.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f691381c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 0x7f690c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f691381c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f690c0b4580 0x7f690c0b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f691381c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0d2330 con 0x7f690c15c740 2023-10-19T10:46:25.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f691381c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0b4a10 con 0x7f690c0b4580 2023-10-19T10:46:25.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f691381c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0b5510 con 0x7f690c0d0750 2023-10-19T10:46:25.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f69115b8640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f690c15c740 0x7f690c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46226/0 (socket says 172.21.15.50:46226) 2023-10-19T10:46:25.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f69115b8640 1 -- 172.21.15.50:0/2330449613 learned_addr learned my addr 172.21.15.50:0/2330449613 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:46:25.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6911db9640 1 --2- 172.21.15.50:0/2330449613 >> v2:172.21.15.183:3300/0 conn(0x7f690c0b4580 0x7f690c0b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6910db7640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 0x7f690c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243440524 0 0) 0x7f6908001110 con 0x7f690c15c740 2023-10-19T10:46:25.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f8003520 con 0x7f690c15c740 2023-10-19T10:46:25.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233993907 0 0) 0x7f68f8003520 con 0x7f690c15c740 2023-10-19T10:46:25.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 msgr2=0x7f690c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 0x7f690c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 >> v2:172.21.15.183:3300/0 conn(0x7f690c0b4580 msgr2=0x7f690c0b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.446+0000 7f6903fff640 1 --2- 172.21.15.50:0/2330449613 >> v2:172.21.15.183:3300/0 conn(0x7f690c0b4580 0x7f690c0b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0d0bd0 con 0x7f690c15c740 2023-10-19T10:46:25.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f6908001800 con 0x7f690c15c740 2023-10-19T10:46:25.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 --2- 172.21.15.50:0/2330449613 >> v2:172.21.15.183:3300/0 conn(0x7f690c0b4580 0x7f690c0b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:46:25.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 >> v1:172.21.15.50:6789/0 conn(0x7f690c15c740 legacy=0x7f690c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8003970 0x7f68f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 0x7f68f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68f8004d00 0x7f68f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0b4a10 con 0x7f68f8004d00 2023-10-19T10:46:25.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0b5510 con 0x7f68f8003970 2023-10-19T10:46:25.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f800b980 con 0x7f690c0d0750 2023-10-19T10:46:25.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68f8004d00 0x7f68f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f69115b8640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8003970 0x7f68f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6910db7640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 0x7f68f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 -- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 msgr2=0x7f68f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c0d0750 0x7f68f80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 -- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8003970 msgr2=0x7f68f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8003970 0x7f68f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.447+0000 7f6911db9640 1 -- 172.21.15.50:0/2330449613 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8003520 con 0x7f68f8004d00 2023-10-19T10:46:25.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f6911db9640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68f8004d00 0x7f68f8005180 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f690400e350 tx=0x7f68f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca481d64dfe68e2d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690402ce00 con 0x7f68f8004d00 2023-10-19T10:46:25.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400f040 con 0x7f68f8004d00 2023-10-19T10:46:25.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f6903fff640 1 -- 172.21.15.50:0/2330449613 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904034640 con 0x7f68f8004d00 2023-10-19T10:46:25.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f691381c640 1 -- 172.21.15.50:0/2330449613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68f8004d00 msgr2=0x7f68f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f691381c640 1 --2- 172.21.15.50:0/2330449613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68f8004d00 0x7f68f8005180 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f690400e350 tx=0x7f68f80057f0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f6911db9640 1 -- 172.21.15.50:0/2330449613 reap_dead start 2023-10-19T10:46:25.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.448+0000 7f691381c640 1 -- 172.21.15.50:0/2330449613 shutdown_connections 2023-10-19T10:46:25.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.449+0000 7f691381c640 1 -- 172.21.15.50:0/2330449613 >> 172.21.15.50:0/2330449613 conn(0x7f690c071320 msgr2=0x7f690c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:25.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.449+0000 7f691381c640 1 -- 172.21.15.50:0/2330449613 shutdown_connections 2023-10-19T10:46:25.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.449+0000 7f691381c640 1 -- 172.21.15.50:0/2330449613 wait complete. 2023-10-19T10:46:25.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.450+0000 7f691381c640 1 Processor -- start 2023-10-19T10:46:25.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.450+0000 7f691381c640 1 -- start start 2023-10-19T10:46:25.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.450+0000 7f691381c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8004d00 0x7f690c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.450+0000 7f691381c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c157ab0 0x7f690c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f691381c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 0x7f690c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f691381c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1595d0 con 0x7f690c0c9690 2023-10-19T10:46:25.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f691381c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c15ee70 con 0x7f68f8004d00 2023-10-19T10:46:25.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f691381c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c15e8b0 con 0x7f690c157ab0 2023-10-19T10:46:25.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 0x7f690c0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 0x7f690c0c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47918/0 (socket says 172.21.15.50:47918) 2023-10-19T10:46:25.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 -- 172.21.15.50:0/2387316745 learned_addr learned my addr 172.21.15.50:0/2387316745 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:25.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f69115b8640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8004d00 0x7f690c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6910db7640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c157ab0 0x7f690c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 -- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c157ab0 msgr2=0x7f690c15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c157ab0 0x7f690c15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 -- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8004d00 msgr2=0x7f690c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8004d00 0x7f690c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 -- 172.21.15.50:0/2387316745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c15eb90 con 0x7f690c0c9690 2023-10-19T10:46:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f69115b8640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8004d00 0x7f690c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:46:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6911db9640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 0x7f690c0c9af0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f690400e350 tx=0x7f6904009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904003b90 con 0x7f690c0c9690 2023-10-19T10:46:25.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6904003cf0 con 0x7f690c0c9690 2023-10-19T10:46:25.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c15d660 con 0x7f690c0c9690 2023-10-19T10:46:25.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.451+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904004260 con 0x7f690c0c9690 2023-10-19T10:46:25.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.452+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c157f30 con 0x7f690c0c9690 2023-10-19T10:46:25.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.453+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c04e420 con 0x7f690c0c9690 2023-10-19T10:46:25.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.453+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f69040043c0 con 0x7f690c0c9690 2023-10-19T10:46:25.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.454+0000 7f6901ffb640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f68f006a050 0x7f68f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.454+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f69040aacf0 con 0x7f690c0c9690 2023-10-19T10:46:25.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.454+0000 7f69115b8640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f68f006a050 0x7f68f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.455+0000 7f69115b8640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f68f006a050 0x7f68f006c510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6908007fd0 tx=0x7f6908007a70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.458+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f6904076580 con 0x7f690c0c9690 2023-10-19T10:46:25.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.633+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_osd_laggy_weight}] v 0) v1 -- 0x7f690c0cda30 con 0x7f690c0c9690 2023-10-19T10:46:25.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.636+0000 7f6901ffb640 1 -- 172.21.15.50:0/2387316745 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_osd_laggy_weight}]=0 v13)=0 v13) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6904009de0 con 0x7f690c0c9690 2023-10-19T10:46:25.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f68f006a050 msgr2=0x7f68f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f68f006a050 0x7f68f006c510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6908007fd0 tx=0x7f6908007a70 comp rx=0 tx=0).stop 2023-10-19T10:46:25.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 msgr2=0x7f690c0c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 0x7f690c0c9af0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f690400e350 tx=0x7f6904009750 comp rx=0 tx=0).stop 2023-10-19T10:46:25.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 shutdown_connections 2023-10-19T10:46:25.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f68f006a050 0x7f68f006c510 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f690c0c9690 0x7f690c0c9af0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f690c157ab0 0x7f690c15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 --2- 172.21.15.50:0/2387316745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68f8004d00 0x7f690c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 >> 172.21.15.50:0/2387316745 conn(0x7f690c071320 msgr2=0x7f690c1e6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:25.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 shutdown_connections 2023-10-19T10:46:25.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.638+0000 7f691381c640 1 -- 172.21.15.50:0/2387316745 wait complete. 2023-10-19T10:46:25.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:46:25.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:46:25.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.757+0000 7f24d192e640 1 Processor -- start 2023-10-19T10:46:25.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.757+0000 7f24d192e640 1 -- start start 2023-10-19T10:46:25.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24d192e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f24cc15c580 0x7f24cc06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24d192e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc06e140 con 0x7f24cc0ccd10 2023-10-19T10:46:25.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24d192e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc15d070 con 0x7f24cc0c9790 2023-10-19T10:46:25.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24d192e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24cc06ec10 con 0x7f24cc15c580 2023-10-19T10:46:25.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24cb7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f24cc0ccd10 0x7f24cc070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46240/0 (socket says 172.21.15.50:46240) 2023-10-19T10:46:25.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24cb7fe640 1 -- 172.21.15.50:0/2959924592 learned_addr learned my addr 172.21.15.50:0/2959924592 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:46:25.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24caffd640 1 --2- 172.21.15.50:0/2959924592 >> v2:172.21.15.183:3300/0 conn(0x7f24cc15c580 0x7f24cc06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732547500 0 0) 0x7f24cc06e140 con 0x7f24cc0ccd10 2023-10-19T10:46:25.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.758+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b8003520 con 0x7f24cc0ccd10 2023-10-19T10:46:25.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1677864413 0 0) 0x7f24b8003520 con 0x7f24cc0ccd10 2023-10-19T10:46:25.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 >> v2:172.21.15.183:3300/0 conn(0x7f24cc15c580 msgr2=0x7f24cc06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 --2- 172.21.15.50:0/2959924592 >> v2:172.21.15.183:3300/0 conn(0x7f24cc15c580 0x7f24cc06dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.766 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 >> v1:172.21.15.183:6789/0 conn(0x7f24cc0c9790 legacy=0x7f24cc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0709b0 con 0x7f24cc0ccd10 2023-10-19T10:46:25.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f24bc001670 con 0x7f24cc0ccd10 2023-10-19T10:46:25.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 >> v1:172.21.15.50:6789/0 conn(0x7f24cc0ccd10 legacy=0x7f24cc070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24b8003970 0x7f24b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.767 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b80047c0 0x7f24b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc06ec10 con 0x7f24b8005180 2023-10-19T10:46:25.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b800c020 con 0x7f24b80047c0 2023-10-19T10:46:25.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b800c1e0 con 0x7f24b8003970 2023-10-19T10:46:25.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24cb7fe640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24caffd640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24b8003970 0x7f24b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24ca7fc640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b80047c0 0x7f24b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24cb7fe640 1 -- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24b8003970 msgr2=0x7f24b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24cb7fe640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24b8003970 0x7f24b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24cb7fe640 1 -- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b80047c0 msgr2=0x7f24b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24cb7fe640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b80047c0 0x7f24b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24cb7fe640 1 -- 172.21.15.50:0/2959924592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8003520 con 0x7f24b8005180 2023-10-19T10:46:25.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.759+0000 7f24ca7fc640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b80047c0 0x7f24b8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:46:25.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.760+0000 7f24cb7fe640 1 -- 172.21.15.50:0/2959924592 reap_dead start 2023-10-19T10:46:25.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.760+0000 7f24cb7fe640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24b800b980 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f24bc001050 tx=0x7f24b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939702086fb352e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.760+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc005ad0 con 0x7f24b8005180 2023-10-19T10:46:25.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.760+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24bc005c30 con 0x7f24b8005180 2023-10-19T10:46:25.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.760+0000 7f24c9ffb640 1 -- 172.21.15.50:0/2959924592 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc02c890 con 0x7f24b8005180 2023-10-19T10:46:25.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 -- 172.21.15.50:0/2959924592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 msgr2=0x7f24b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24b800b980 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f24bc001050 tx=0x7f24b80057f0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 -- 172.21.15.50:0/2959924592 shutdown_connections 2023-10-19T10:46:25.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 --2- 172.21.15.50:0/2959924592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24b800b980 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 -- 172.21.15.50:0/2959924592 >> 172.21.15.50:0/2959924592 conn(0x7f24cc0cbbc0 msgr2=0x7f24cc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:25.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 -- 172.21.15.50:0/2959924592 shutdown_connections 2023-10-19T10:46:25.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.761+0000 7f24d192e640 1 -- 172.21.15.50:0/2959924592 wait complete. 2023-10-19T10:46:25.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.762+0000 7f24d192e640 1 Processor -- start 2023-10-19T10:46:25.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.762+0000 7f24d192e640 1 -- start start 2023-10-19T10:46:25.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24d192e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24cc0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24d192e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24cc158810 0x7f24cc0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24d192e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24cc0ae080 0x7f24cc0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24d192e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0ab8e0 con 0x7f24b8005180 2023-10-19T10:46:25.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24d192e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0ae520 con 0x7f24cc158810 2023-10-19T10:46:25.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24d192e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0abb30 con 0x7f24cc0ae080 2023-10-19T10:46:25.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24cc0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24cc0b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47948/0 (socket says 172.21.15.50:47948) 2023-10-19T10:46:25.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 -- 172.21.15.50:0/3697805824 learned_addr learned my addr 172.21.15.50:0/3697805824 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:25.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 -- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24cc0ae080 msgr2=0x7f24cc0c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24cc0ae080 0x7f24cc0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 -- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24cc158810 msgr2=0x7f24cc0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24cc158810 0x7f24cc0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.763+0000 7f24caffd640 1 -- 172.21.15.50:0/3697805824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0abe10 con 0x7f24b8005180 2023-10-19T10:46:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.764+0000 7f24caffd640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24cc0b2900 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009f90 tx=0x7f24c400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.764+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c400e040 con 0x7f24b8005180 2023-10-19T10:46:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.764+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c400f040 con 0x7f24b8005180 2023-10-19T10:46:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.764+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4015870 con 0x7f24b8005180 2023-10-19T10:46:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.764+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24cc0ac0f0 con 0x7f24b8005180 2023-10-19T10:46:25.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.764+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc0c7090 con 0x7f24b8005180 2023-10-19T10:46:25.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.765+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc073990 con 0x7f24b8005180 2023-10-19T10:46:25.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.765+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f24c40159d0 con 0x7f24b8005180 2023-10-19T10:46:25.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.766+0000 7f24c3fff640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f24ac06a080 0x7f24ac06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.766+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f24c4089500 con 0x7f24b8005180 2023-10-19T10:46:25.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.766+0000 7f24ca7fc640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f24ac06a080 0x7f24ac06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.766+0000 7f24ca7fc640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f24ac06a080 0x7f24ac06c540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0b0ac0 tx=0x7f24b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.770+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f24c4058460 con 0x7f24b8005180 2023-10-19T10:46:25.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 Processor -- start 2023-10-19T10:46:25.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 -- start start 2023-10-19T10:46:25.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0fe415c680 0x7f0fe40d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0fe40cd960 0x7f0fe40d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe40cde90 con 0x7f0fe40d0e70 2023-10-19T10:46:25.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40d1300 con 0x7f0fe415c680 2023-10-19T10:46:25.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fea2dc640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40ce170 con 0x7f0fe40cd960 2023-10-19T10:46:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.781+0000 7f0fe3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0fe415c680 0x7f0fe40d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0fe415c680 0x7f0fe40d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47950/0 (socket says 172.21.15.50:47950) 2023-10-19T10:46:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe3fff640 1 -- 172.21.15.50:0/414292901 learned_addr learned my addr 172.21.15.50:0/414292901 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe37fe640 1 --2- 172.21.15.50:0/414292901 >> v2:172.21.15.183:3300/0 conn(0x7f0fe40cd960 0x7f0fe40d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe3fff640 1 -- 172.21.15.50:0/414292901 >> v2:172.21.15.183:3300/0 conn(0x7f0fe40cd960 msgr2=0x7f0fe40d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe3fff640 1 --2- 172.21.15.50:0/414292901 >> v2:172.21.15.183:3300/0 conn(0x7f0fe40cd960 0x7f0fe40d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe3fff640 1 -- 172.21.15.50:0/414292901 >> v1:172.21.15.183:6789/0 conn(0x7f0fe40d0e70 legacy=0x7f0fe40ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:46:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.782+0000 7f0fe3fff640 1 -- 172.21.15.50:0/414292901 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40b1c80 con 0x7f0fe415c680 2023-10-19T10:46:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fe3fff640 1 --2- 172.21.15.50:0/414292901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40d0090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b8420 tx=0x7f0fd402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5439ca9f6b658d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fe2ffd640 1 -- 172.21.15.50:0/414292901 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004230 con 0x7f0fe415c680 2023-10-19T10:46:25.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fe2ffd640 1 -- 172.21.15.50:0/414292901 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd4004390 con 0x7f0fe415c680 2023-10-19T10:46:25.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fe2ffd640 1 -- 172.21.15.50:0/414292901 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4030840 con 0x7f0fe415c680 2023-10-19T10:46:25.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 -- 172.21.15.50:0/414292901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 msgr2=0x7f0fe40d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 --2- 172.21.15.50:0/414292901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40d0090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b8420 tx=0x7f0fd402b980 comp rx=0 tx=0).stop 2023-10-19T10:46:25.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 -- 172.21.15.50:0/414292901 shutdown_connections 2023-10-19T10:46:25.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 --2- 172.21.15.50:0/414292901 >> v2:172.21.15.183:3300/0 conn(0x7f0fe40cd960 0x7f0fe40d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 --2- 172.21.15.50:0/414292901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40d0090 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 -- 172.21.15.50:0/414292901 >> 172.21.15.50:0/414292901 conn(0x7f0fe40d2370 msgr2=0x7f0fe40d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:25.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 -- 172.21.15.50:0/414292901 shutdown_connections 2023-10-19T10:46:25.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.783+0000 7f0fea2dc640 1 -- 172.21.15.50:0/414292901 wait complete. 2023-10-19T10:46:25.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.785+0000 7f0fea2dc640 1 Processor -- start 2023-10-19T10:46:25.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.785+0000 7f0fea2dc640 1 -- start start 2023-10-19T10:46:25.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fe40cd960 0x7f0fe4158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe40d0e70 0x7f0fe415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415fd50 con 0x7f0fe415c680 2023-10-19T10:46:25.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415e1e0 con 0x7f0fe40d0e70 2023-10-19T10:46:25.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415e4c0 con 0x7f0fe40cd960 2023-10-19T10:46:25.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47956/0 (socket says 172.21.15.50:47956) 2023-10-19T10:46:25.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 -- 172.21.15.50:0/306600763 learned_addr learned my addr 172.21.15.50:0/306600763 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:46:25.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe3fff640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fe40cd960 0x7f0fe4158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe37fe640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe40d0e70 0x7f0fe415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 -- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fe40cd960 msgr2=0x7f0fe4158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0fe40cd960 0x7f0fe4158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 -- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe40d0e70 msgr2=0x7f0fe415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe40d0e70 0x7f0fe415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 -- 172.21.15.50:0/306600763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009460 con 0x7f0fe415c680 2023-10-19T10:46:25.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe37fe640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe40d0e70 0x7f0fe415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:46:25.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe8879640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fe415c680 0x7f0fe40c7eb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00c780 tx=0x7f0fdc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc004b60 con 0x7f0fe415c680 2023-10-19T10:46:25.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00ce90 con 0x7f0fe415c680 2023-10-19T10:46:25.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.786+0000 7f0fea2dc640 1 -- 172.21.15.50:0/306600763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe415e7a0 con 0x7f0fe415c680 2023-10-19T10:46:25.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.787+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00a850 con 0x7f0fe415c680 2023-10-19T10:46:25.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.787+0000 7f0fea2dc640 1 -- 172.21.15.50:0/306600763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe409fd20 con 0x7f0fe415c680 2023-10-19T10:46:25.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.787+0000 7f0fea2dc640 1 -- 172.21.15.50:0/306600763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f0fe415ecc0 con 0x7f0fe415c680 2023-10-19T10:46:25.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.788+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0fdc00a9b0 con 0x7f0fe415c680 2023-10-19T10:46:25.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.788+0000 7f0fe0ff9640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fc406a050 0x7f0fc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:46:25.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.789+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0fdc089280 con 0x7f0fe415c680 2023-10-19T10:46:25.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.789+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+701 (secure 0 0 0) 0x7f0fdc089650 con 0x7f0fe415c680 2023-10-19T10:46:25.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.789+0000 7f0fe3fff640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fc406a050 0x7f0fc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:46:25.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.789+0000 7f0fea2dc640 1 -- 172.21.15.50:0/306600763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f0fe415ecc0 con 0x7f0fe415c680 2023-10-19T10:46:25.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.789+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 7 ==== log(2 entries from seq 535 at 2023-10-19T10:46:24.313454+0000) v1 ==== 694+0+0 (secure 0 0 0) 0x7f0fdc058aa0 con 0x7f0fe415c680 2023-10-19T10:46:25.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.789+0000 7f0fe3fff640 1 --2- 172.21.15.50:0/306600763 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0fc406a050 0x7f0fc406c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4035000 tx=0x7f0fd4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:46:25.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.983+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24cc0c94d0 con 0x7f24b8005180 2023-10-19T10:46:25.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.984+0000 7f24c3fff640 1 -- 172.21.15.50:0/3697805824 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f24c400fea0 con 0x7f24b8005180 2023-10-19T10:46:25.987 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:46:25.987 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:45:45.408771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":112,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:46:25.988 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 16 2023-10-19T10:46:25.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f24ac06a080 msgr2=0x7f24ac06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f24ac06a080 0x7f24ac06c540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0b0ac0 tx=0x7f24b4009040 comp rx=0 tx=0).stop 2023-10-19T10:46:25.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 msgr2=0x7f24cc0b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:46:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24cc0b2900 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009f90 tx=0x7f24c400f580 comp rx=0 tx=0).stop 2023-10-19T10:46:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 shutdown_connections 2023-10-19T10:46:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f24ac06a080 0x7f24ac06c540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24cc0ae080 0x7f24cc0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.990 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24cc158810 0x7f24cc0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 --2- 172.21.15.50:0/3697805824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24b8005180 0x7f24cc0b2900 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:46:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 >> 172.21.15.50:0/3697805824 conn(0x7f24cc0cbbc0 msgr2=0x7f24cc0aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:46:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 shutdown_connections 2023-10-19T10:46:25.991 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:25.986+0000 7f24d192e640 1 -- 172.21.15.50:0/3697805824 wait complete. 2023-10-19T10:46:25.999 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:46:26.122 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":465.65621752200002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.34099502781621083,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}},{"id":5994,"entity":{"name":{"type":"client","num":5994},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":216420809}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":467.24517742900002,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.33974543583933509,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5994 192.168.0.1:0/216420809","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"0","hostname":"smithi050","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"105851","root":"/"}}] 2023-10-19T10:46:26.123 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.3409950278162108 }, "state": "open", "uptime": 465.656217522 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi050", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "105851", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 216420809, "type": "any" }, "name": { "num": 5994, "type": "client" } }, "id": 5994, "inst": "client.5994 192.168.0.1:0/216420809", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.3397454358393351 }, "state": "stale", "uptime": 467.245177429 } ] 2023-10-19T10:46:26.224 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:46:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:26.331+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 8 ==== log(7 entries from seq 538 at 2023-10-19T10:46:25.318046+0000) v1 ==== 1668+0+0 (secure 0 0 0) 0x7f0fdc05c1d0 con 0x7f0fe415c680 2023-10-19T10:46:26.335 INFO:teuthology.orchestra.run.smithi050.stdout: cluster: 2023-10-19T10:46:26.335 INFO:teuthology.orchestra.run.smithi050.stdout: id: 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:46:26.335 INFO:teuthology.orchestra.run.smithi050.stdout: health: HEALTH_WARN 2023-10-19T10:46:26.335 INFO:teuthology.orchestra.run.smithi050.stdout: 1 client(s) laggy due to laggy OSDs 2023-10-19T10:46:26.336 INFO:teuthology.orchestra.run.smithi050.stdout: Reduced data availability: 5 pgs inactive, 12 pgs peering 2023-10-19T10:46:26.336 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:46:26.336 INFO:teuthology.orchestra.run.smithi050.stdout: services: 2023-10-19T10:46:26.336 INFO:teuthology.orchestra.run.smithi050.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2023-10-19T10:46:26.336 INFO:teuthology.orchestra.run.smithi050.stdout: mgr: x(active, since 9m), standbys: y 2023-10-19T10:46:26.337 INFO:teuthology.orchestra.run.smithi050.stdout: mds: 1/1 daemons up, 3 standby 2023-10-19T10:46:26.337 INFO:teuthology.orchestra.run.smithi050.stdout: osd: 8 osds: 8 up (since 0.471484s), 8 in (since 9m) 2023-10-19T10:46:26.337 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:46:26.337 INFO:teuthology.orchestra.run.smithi050.stdout: data: 2023-10-19T10:46:26.337 INFO:teuthology.orchestra.run.smithi050.stdout: volumes: 1/1 healthy 2023-10-19T10:46:26.338 INFO:teuthology.orchestra.run.smithi050.stdout: pools: 4 pools, 105 pgs 2023-10-19T10:46:26.338 INFO:teuthology.orchestra.run.smithi050.stdout: objects: 24 objects, 581 KiB 2023-10-19T10:46:26.338 INFO:teuthology.orchestra.run.smithi050.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2023-10-19T10:46:26.338 INFO:teuthology.orchestra.run.smithi050.stdout: pgs: 16.190% pgs not active 2023-10-19T10:46:26.341 INFO:teuthology.orchestra.run.smithi050.stdout: 83 active+clean 2023-10-19T10:46:26.342 INFO:teuthology.orchestra.run.smithi050.stdout: 17 peering 2023-10-19T10:46:26.342 INFO:teuthology.orchestra.run.smithi050.stdout: 4 active+undersized 2023-10-19T10:46:26.342 INFO:teuthology.orchestra.run.smithi050.stdout: 1 stale+active+clean 2023-10-19T10:46:26.342 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:46:26.343 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:46:26.343 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:25.318046+0000 mon.a [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2023-10-19T10:46:26.343 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:25.321415+0000 mon.a [INF] osd.2 [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] boot 2023-10-19T10:46:26.343 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:25.724426+0000 mds.a [WRN] evicting unresponsive client smithi050:0 (5994), after 345.832 seconds 2023-10-19T10:46:26.344 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:25.724434+0000 mds.a [INF] Evicting (and blocklisting) client session 5994 (192.168.0.1:0/216420809) 2023-10-19T10:46:27.698 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:46:27.701 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:46:29.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:29.335+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 9 ==== log(1 entries from seq 547 at 2023-10-19T10:46:28.647310+0000) v1 ==== 272+0+0 (secure 0 0 0) 0x7f0fdc0567b0 con 0x7f0fe415c680 2023-10-19T10:46:29.338 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:28.647310+0000 mon.a [INF] MDS health message cleared (mds.?): Client 5994 is laggy; not evicted because some OSD(s) is/are laggy 2023-10-19T10:46:30.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:30.336+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 10 ==== log(2 entries from seq 548 at 2023-10-19T10:46:29.335001+0000) v1 ==== 479+0+0 (secure 0 0 0) 0x7f0fdc054b20 con 0x7f0fe415c680 2023-10-19T10:46:30.340 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:29.335001+0000 mon.a [INF] Health check cleared: MDS_CLIENTS_LAGGY (was: 1 client(s) laggy due to laggy OSDs) 2023-10-19T10:46:30.340 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:30.205689+0000 mon.a [WRN] Health check update: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2023-10-19T10:46:31.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:46:31.388+0000 7f0fe0ff9640 1 -- 172.21.15.50:0/306600763 <== mon.0 v2:172.21.15.50:3300/0 11 ==== log(2 entries from seq 552 at 2023-10-19T10:46:31.337336+0000) v1 ==== 425+0+0 (secure 0 0 0) 0x7f0fdc01a3a0 con 0x7f0fe415c680 2023-10-19T10:46:31.392 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:31.337336+0000 mon.a [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2023-10-19T10:46:31.392 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:46:31.337370+0000 mon.a [INF] Cluster is now healthy 2023-10-19T10:46:57.835 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:46:57.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:47:26.238 WARNING:tasks.cephfs.test_client_recovery:1 != 2 : Expected 1 sessions, found 2, retrying 2023-10-19T10:47:27.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:47:27.244 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:47:27.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.359+0000 7f00b3957640 1 Processor -- start 2023-10-19T10:47:27.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.360+0000 7f00b3957640 1 -- start start 2023-10-19T10:47:27.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.360+0000 7f00b3957640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00ac0d0e30 0x7f00ac15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.360+0000 7f00b3957640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac0cd920 0x7f00ac0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.360+0000 7f00b3957640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac06aa30 con 0x7f00ac15c570 2023-10-19T10:47:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.360+0000 7f00b3957640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac15ce90 con 0x7f00ac0d0e30 2023-10-19T10:47:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.360+0000 7f00b3957640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac06b2d0 con 0x7f00ac0cd920 2023-10-19T10:47:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b0ef2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00ac0d0e30 0x7f00ac15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b1ef4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac0cd920 0x7f00ac0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b16f3640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f00ac15c570 0x7f00ac06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:52198/0 (socket says 172.21.15.50:52198) 2023-10-19T10:47:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b16f3640 1 -- 172.21.15.50:0/2564753536 learned_addr learned my addr 172.21.15.50:0/2564753536 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:47:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b1ef4640 1 -- 172.21.15.50:0/2564753536 >> v2:172.21.15.183:3300/0 conn(0x7f00ac0d0e30 msgr2=0x7f00ac15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:47:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b1ef4640 1 --2- 172.21.15.50:0/2564753536 >> v2:172.21.15.183:3300/0 conn(0x7f00ac0d0e30 0x7f00ac15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b1ef4640 1 -- 172.21.15.50:0/2564753536 >> v1:172.21.15.183:6789/0 conn(0x7f00ac15c570 legacy=0x7f00ac06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b1ef4640 1 -- 172.21.15.50:0/2564753536 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0d1270 con 0x7f00ac0cd920 2023-10-19T10:47:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.361+0000 7f00b0ef2640 1 --2- 172.21.15.50:0/2564753536 >> v2:172.21.15.183:3300/0 conn(0x7f00ac0d0e30 0x7f00ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:47:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.363+0000 7f00b1ef4640 1 --2- 172.21.15.50:0/2564753536 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac0cd920 0x7f00ac0d26a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009d90 tx=0x7f00a402bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd67c27c18b7d8b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:47:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.363+0000 7f009ffff640 1 -- 172.21.15.50:0/2564753536 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a402c980 con 0x7f00ac0cd920 2023-10-19T10:47:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.363+0000 7f009ffff640 1 -- 172.21.15.50:0/2564753536 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a402cae0 con 0x7f00ac0cd920 2023-10-19T10:47:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.363+0000 7f009ffff640 1 -- 172.21.15.50:0/2564753536 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a402ce70 con 0x7f00ac0cd920 2023-10-19T10:47:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.363+0000 7f00b3957640 1 -- 172.21.15.50:0/2564753536 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac0cd920 msgr2=0x7f00ac0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.363+0000 7f00b3957640 1 --2- 172.21.15.50:0/2564753536 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac0cd920 0x7f00ac0d26a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009d90 tx=0x7f00a402bc50 comp rx=0 tx=0).stop 2023-10-19T10:47:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.364+0000 7f00b3957640 1 -- 172.21.15.50:0/2564753536 shutdown_connections 2023-10-19T10:47:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.364+0000 7f00b3957640 1 --2- 172.21.15.50:0/2564753536 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac0cd920 0x7f00ac0d26a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.364+0000 7f00b3957640 1 --2- 172.21.15.50:0/2564753536 >> v2:172.21.15.183:3300/0 conn(0x7f00ac0d0e30 0x7f00ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.364+0000 7f00b3957640 1 -- 172.21.15.50:0/2564753536 >> 172.21.15.50:0/2564753536 conn(0x7f00ac089ff0 msgr2=0x7f00ac09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:47:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.364+0000 7f00b3957640 1 -- 172.21.15.50:0/2564753536 shutdown_connections 2023-10-19T10:47:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.364+0000 7f00b3957640 1 -- 172.21.15.50:0/2564753536 wait complete. 2023-10-19T10:47:27.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.365+0000 7f00b3957640 1 Processor -- start 2023-10-19T10:47:27.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.365+0000 7f00b3957640 1 -- start start 2023-10-19T10:47:27.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b3957640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ac0cd920 0x7f00ac15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b3957640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 0x7f00ac0b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b3957640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac15c570 0x7f00ac0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b3957640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15d070 con 0x7f00ac0d0e30 2023-10-19T10:47:27.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b3957640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15d310 con 0x7f00ac0cd920 2023-10-19T10:47:27.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b3957640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15d5f0 con 0x7f00ac15c570 2023-10-19T10:47:27.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 0x7f00ac0b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 0x7f00ac0b79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36140/0 (socket says 172.21.15.50:36140) 2023-10-19T10:47:27.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 -- 172.21.15.50:0/1354833475 learned_addr learned my addr 172.21.15.50:0/1354833475 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:47:27.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b16f3640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ac0cd920 0x7f00ac15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b1ef4640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac15c570 0x7f00ac0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 -- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac15c570 msgr2=0x7f00ac0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac15c570 0x7f00ac0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 -- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ac0cd920 msgr2=0x7f00ac15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ac0cd920 0x7f00ac15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 -- 172.21.15.50:0/1354833475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8009400 con 0x7f00ac0d0e30 2023-10-19T10:47:27.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.366+0000 7f00b0ef2640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 0x7f00ac0b79d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00a800e960 tx=0x7f00a800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:47:27.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.367+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a800ca00 con 0x7f00ac0d0e30 2023-10-19T10:47:27.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.367+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a800cb60 con 0x7f00ac0d0e30 2023-10-19T10:47:27.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.367+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a4009a70 con 0x7f00ac0d0e30 2023-10-19T10:47:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.367+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8013470 con 0x7f00ac0d0e30 2023-10-19T10:47:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.367+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac15dbb0 con 0x7f00ac0d0e30 2023-10-19T10:47:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.368+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac073990 con 0x7f00ac0d0e30 2023-10-19T10:47:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.368+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f00a80135d0 con 0x7f00ac0d0e30 2023-10-19T10:47:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.369+0000 7f009dffb640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f009006a050 0x7f009006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.369+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f00a8088950 con 0x7f00ac0d0e30 2023-10-19T10:47:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.369+0000 7f00b16f3640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f009006a050 0x7f009006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.370+0000 7f00b16f3640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f009006a050 0x7f009006c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00a0000fb0 tx=0x7f00a0007e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:47:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.374+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f00a8053f40 con 0x7f00ac0d0e30 2023-10-19T10:47:27.386 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.384+0000 7fdd80bcb640 1 Processor -- start 2023-10-19T10:47:27.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.384+0000 7fdd80bcb640 1 -- start start 2023-10-19T10:47:27.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd80bcb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c15c680 0x7fdd7c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd80bcb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdd7c0cd960 0x7fdd7c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd80bcb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c0cde90 con 0x7fdd7c0d0e70 2023-10-19T10:47:27.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd80bcb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c0d1300 con 0x7fdd7c0cd960 2023-10-19T10:47:27.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd80bcb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0ce170 con 0x7fdd7c15c680 2023-10-19T10:47:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd79d9b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdd7c0cd960 0x7fdd7c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd7a59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c15c680 0x7fdd7c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd7ad9d640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fdd7c0d0e70 0x7fdd7c0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:52206/0 (socket says 172.21.15.50:52206) 2023-10-19T10:47:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.385+0000 7fdd7ad9d640 1 -- 172.21.15.50:0/1086132114 learned_addr learned my addr 172.21.15.50:0/1086132114 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:47:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd79d9b640 1 -- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c15c680 msgr2=0x7fdd7c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:47:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd79d9b640 1 --2- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c15c680 0x7fdd7c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd79d9b640 1 -- 172.21.15.50:0/1086132114 >> v1:172.21.15.183:6789/0 conn(0x7fdd7c0d0e70 legacy=0x7fdd7c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd7959a640 1 -- 172.21.15.50:0/1086132114 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921235266 0 0) 0x7fdd7c0cde90 con 0x7fdd7c0d0e70 2023-10-19T10:47:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd79d9b640 1 -- 172.21.15.50:0/1086132114 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0b1c80 con 0x7fdd7c0cd960 2023-10-19T10:47:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd7a59c640 1 --2- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c15c680 0x7fdd7c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:47:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.386+0000 7fdd79d9b640 1 --2- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0d0930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097a0 tx=0x7fdd7002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158c0e89e5ed29a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:47:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd7959a640 1 -- 172.21.15.50:0/1086132114 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70030610 con 0x7fdd7c0cd960 2023-10-19T10:47:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd7959a640 1 -- 172.21.15.50:0/1086132114 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70030770 con 0x7fdd7c0cd960 2023-10-19T10:47:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd7959a640 1 -- 172.21.15.50:0/1086132114 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70030b00 con 0x7fdd7c0cd960 2023-10-19T10:47:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 -- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c0cd960 msgr2=0x7fdd7c0d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 --2- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0d0930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097a0 tx=0x7fdd7002b960 comp rx=0 tx=0).stop 2023-10-19T10:47:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 -- 172.21.15.50:0/1086132114 shutdown_connections 2023-10-19T10:47:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 --2- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0d0930 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 --2- 172.21.15.50:0/1086132114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c15c680 0x7fdd7c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 -- 172.21.15.50:0/1086132114 >> 172.21.15.50:0/1086132114 conn(0x7fdd7c0d2370 msgr2=0x7fdd7c0d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:47:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 -- 172.21.15.50:0/1086132114 shutdown_connections 2023-10-19T10:47:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.387+0000 7fdd80bcb640 1 -- 172.21.15.50:0/1086132114 wait complete. 2023-10-19T10:47:27.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.389+0000 7fdd80bcb640 1 Processor -- start 2023-10-19T10:47:27.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.389+0000 7fdd80bcb640 1 -- start start 2023-10-19T10:47:27.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.389+0000 7fdd80bcb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.397 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.389+0000 7fdd80bcb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c0d0e70 0x7fdd7c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.389+0000 7fdd80bcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c15c680 0x7fdd7c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd80bcb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0c5f40 con 0x7fdd7c0cd960 2023-10-19T10:47:27.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd80bcb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0cba70 con 0x7fdd7c15c680 2023-10-19T10:47:27.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd80bcb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0cbd50 con 0x7fdd7c0d0e70 2023-10-19T10:47:27.398 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36150/0 (socket says 172.21.15.50:36150) 2023-10-19T10:47:27.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 -- 172.21.15.50:0/3556459066 learned_addr learned my addr 172.21.15.50:0/3556459066 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:47:27.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7ad9d640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c15c680 0x7fdd7c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd79d9b640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c0d0e70 0x7fdd7c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.399 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 -- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c0d0e70 msgr2=0x7fdd7c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd7c0d0e70 0x7fdd7c0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 -- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c15c680 msgr2=0x7fdd7c157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd7c15c680 0x7fdd7c157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 -- 172.21.15.50:0/3556459066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0093a0 con 0x7fdd7c0cd960 2023-10-19T10:47:27.400 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd7a59c640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd7c0cd960 0x7fdd7c0c94b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00e9b0 tx=0x7fdd6c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:47:27.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd6affd640 1 -- 172.21.15.50:0/3556459066 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c005990 con 0x7fdd7c0cd960 2023-10-19T10:47:27.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd6affd640 1 -- 172.21.15.50:0/3556459066 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c005af0 con 0x7fdd7c0cd960 2023-10-19T10:47:27.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd80bcb640 1 -- 172.21.15.50:0/3556459066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd70009400 con 0x7fdd7c0cd960 2023-10-19T10:47:27.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd6affd640 1 -- 172.21.15.50:0/3556459066 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c005e80 con 0x7fdd7c0cd960 2023-10-19T10:47:27.401 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.390+0000 7fdd80bcb640 1 -- 172.21.15.50:0/3556459066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c0cc310 con 0x7fdd7c0cd960 2023-10-19T10:47:27.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.391+0000 7fdd80bcb640 1 -- 172.21.15.50:0/3556459066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdd7c066e80 con 0x7fdd7c0cd960 2023-10-19T10:47:27.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.392+0000 7fdd6affd640 1 -- 172.21.15.50:0/3556459066 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd6c00c9a0 con 0x7fdd7c0cd960 2023-10-19T10:47:27.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.392+0000 7fdd6affd640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdd5c06a050 0x7fdd5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:47:27.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.392+0000 7fdd6affd640 1 -- 172.21.15.50:0/3556459066 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fdd6c0891a0 con 0x7fdd7c0cd960 2023-10-19T10:47:27.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.393+0000 7fdd79d9b640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdd5c06a050 0x7fdd5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:47:27.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.393+0000 7fdd6affd640 1 -- 172.21.15.50:0/3556459066 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+450 (secure 0 0 0) 0x7fdd6c089610 con 0x7fdd7c0cd960 2023-10-19T10:47:27.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.393+0000 7fdd80bcb640 1 -- 172.21.15.50:0/3556459066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fdd7c0cf560 con 0x7fdd7c0cd960 2023-10-19T10:47:27.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.393+0000 7fdd79d9b640 1 --2- 172.21.15.50:0/3556459066 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fdd5c06a050 0x7fdd5c06c510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdd70036000 tx=0x7fdd70008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:47:27.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.591+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00ac0d3480 con 0x7f00ac0d0e30 2023-10-19T10:47:27.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.591+0000 7f009dffb640 1 -- 172.21.15.50:0/1354833475 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f00a8057560 con 0x7f00ac0d0e30 2023-10-19T10:47:27.595 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:47:27.596 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:47:27.597 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:47:27.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.593+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f009006a050 msgr2=0x7f009006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.593+0000 7f00b3957640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f009006a050 0x7f009006c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00a0000fb0 tx=0x7f00a0007e70 comp rx=0 tx=0).stop 2023-10-19T10:47:27.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.593+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 msgr2=0x7f00ac0b79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:47:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.593+0000 7f00b3957640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 0x7f00ac0b79d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00a800e960 tx=0x7f00a800ee20 comp rx=0 tx=0).stop 2023-10-19T10:47:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 shutdown_connections 2023-10-19T10:47:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f009006a050 0x7f009006c510 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00ac15c570 0x7f00ac0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00ac0d0e30 0x7f00ac0b79d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 --2- 172.21.15.50:0/1354833475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ac0cd920 0x7f00ac15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:47:27.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 >> 172.21.15.50:0/1354833475 conn(0x7f00ac089ff0 msgr2=0x7f00ac15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:47:27.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 shutdown_connections 2023-10-19T10:47:27.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:47:27.594+0000 7f00b3957640 1 -- 172.21.15.50:0/1354833475 wait complete. 2023-10-19T10:47:27.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:47:27.730 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":527.26563165799996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.29575160320205768,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}}] 2023-10-19T10:47:27.731 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2957516032020577 }, "state": "open", "uptime": 527.265631658 } ] 2023-10-19T10:47:27.832 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:47:27.982 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:47:27.985 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:47:58.146 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:47:58.149 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:48:27.846 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:48:27.847 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-19T10:48:28.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:48:28.211 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:48:28.849 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:48:28.853 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:48:28.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.968+0000 7fb3548b1640 1 Processor -- start 2023-10-19T10:48:28.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.968+0000 7fb3548b1640 1 -- start start 2023-10-19T10:48:28.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb3548b1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb3500cc550 0x7fb350160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb3548b1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb3500ab8d0 0x7fb3500abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb3548b1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35006c2e0 con 0x7fb35006fea0 2023-10-19T10:48:28.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb3548b1640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35006b140 con 0x7fb3500ab8d0 2023-10-19T10:48:28.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb3548b1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3501610a0 con 0x7fb3500cc550 2023-10-19T10:48:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb3500ab8d0 0x7fb3500abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb3500ab8d0 0x7fb3500abd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34972/0 (socket says 172.21.15.50:34972) 2023-10-19T10:48:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34ed9d640 1 -- 172.21.15.50:0/2924298908 learned_addr learned my addr 172.21.15.50:0/2924298908 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:48:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34e59c640 1 --2- 172.21.15.50:0/2924298908 >> v2:172.21.15.183:3300/0 conn(0x7fb3500cc550 0x7fb350160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:28.977 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34ed9d640 1 -- 172.21.15.50:0/2924298908 >> v2:172.21.15.183:3300/0 conn(0x7fb3500cc550 msgr2=0x7fb350160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:48:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/2924298908 >> v2:172.21.15.183:3300/0 conn(0x7fb3500cc550 0x7fb350160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.969+0000 7fb34ed9d640 1 -- 172.21.15.50:0/2924298908 >> v1:172.21.15.183:6789/0 conn(0x7fb35006fea0 legacy=0x7fb3500cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:48:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.970+0000 7fb34ed9d640 1 -- 172.21.15.50:0/2924298908 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500703d0 con 0x7fb3500ab8d0 2023-10-19T10:48:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.970+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/2924298908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500ab8d0 0x7fb3500abd80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb350065710 tx=0x7fb34802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb1e31ff2a5b865 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:48:28.978 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.970+0000 7fb34d59a640 1 -- 172.21.15.50:0/2924298908 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34802ca80 con 0x7fb3500ab8d0 2023-10-19T10:48:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.970+0000 7fb34d59a640 1 -- 172.21.15.50:0/2924298908 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34802cbe0 con 0x7fb3500ab8d0 2023-10-19T10:48:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb34d59a640 1 -- 172.21.15.50:0/2924298908 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480343f0 con 0x7fb3500ab8d0 2023-10-19T10:48:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 -- 172.21.15.50:0/2924298908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500ab8d0 msgr2=0x7fb3500abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:28.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 --2- 172.21.15.50:0/2924298908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500ab8d0 0x7fb3500abd80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb350065710 tx=0x7fb34802bee0 comp rx=0 tx=0).stop 2023-10-19T10:48:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 -- 172.21.15.50:0/2924298908 shutdown_connections 2023-10-19T10:48:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 --2- 172.21.15.50:0/2924298908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500ab8d0 0x7fb3500abd80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 --2- 172.21.15.50:0/2924298908 >> v2:172.21.15.183:3300/0 conn(0x7fb3500cc550 0x7fb350160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 -- 172.21.15.50:0/2924298908 >> 172.21.15.50:0/2924298908 conn(0x7fb3500ce980 msgr2=0x7fb3500ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:48:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 -- 172.21.15.50:0/2924298908 shutdown_connections 2023-10-19T10:48:28.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.971+0000 7fb3548b1640 1 -- 172.21.15.50:0/2924298908 wait complete. 2023-10-19T10:48:28.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.973+0000 7fb3548b1640 1 Processor -- start 2023-10-19T10:48:28.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.973+0000 7fb3548b1640 1 -- start start 2023-10-19T10:48:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.973+0000 7fb3548b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb35006fea0 0x7fb3500ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.973+0000 7fb3548b1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3500ab8d0 0x7fb35014f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb3548b1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 0x7fb35015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb3548b1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500ae600 con 0x7fb3500cc550 2023-10-19T10:48:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb3548b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500aeb80 con 0x7fb35006fea0 2023-10-19T10:48:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb3548b1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500ae8a0 con 0x7fb3500ab8d0 2023-10-19T10:48:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 0x7fb35015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 0x7fb35015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34984/0 (socket says 172.21.15.50:34984) 2023-10-19T10:48:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 -- 172.21.15.50:0/134675549 learned_addr learned my addr 172.21.15.50:0/134675549 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:48:28.985 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34e59c640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb35006fea0 0x7fb3500ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:28.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34dd9b640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3500ab8d0 0x7fb35014f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:28.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 -- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3500ab8d0 msgr2=0x7fb35014f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:28.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3500ab8d0 0x7fb35014f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:28.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 -- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb35006fea0 msgr2=0x7fb3500ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:28.986 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb35006fea0 0x7fb3500ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:28.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 -- 172.21.15.50:0/134675549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348009b30 con 0x7fb3500cc550 2023-10-19T10:48:28.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 0x7fb35015e000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb348009f40 tx=0x7fb348004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:48:28.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348009510 con 0x7fb3500cc550 2023-10-19T10:48:28.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34803e450 con 0x7fb3500cc550 2023-10-19T10:48:28.987 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3500ac400 con 0x7fb3500cc550 2023-10-19T10:48:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.974+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34803e7e0 con 0x7fb3500cc550 2023-10-19T10:48:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.975+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3500ac9c0 con 0x7fb3500cc550 2023-10-19T10:48:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.976+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3500acbb0 con 0x7fb3500cc550 2023-10-19T10:48:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.976+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb34802cd50 con 0x7fb3500cc550 2023-10-19T10:48:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.976+0000 7fb33effd640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb33006a050 0x7fb33006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.977+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb3480ab140 con 0x7fb3500cc550 2023-10-19T10:48:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.977+0000 7fb34e59c640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb33006a050 0x7fb33006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.977+0000 7fb34e59c640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb33006a050 0x7fb33006c510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb3400079a0 tx=0x7fb340008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:48:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.981+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb348076850 con 0x7fb3500cc550 2023-10-19T10:48:28.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.993+0000 7ffac8c0b640 1 Processor -- start 2023-10-19T10:48:28.996 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.993+0000 7ffac8c0b640 1 -- start start 2023-10-19T10:48:28.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac8c0b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffac406a810 0x7ffac40add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac8c0b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac406abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac8c0b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffac415ddd0 0x7ffac415e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:28.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac8c0b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac406e870 con 0x7ffac406a810 2023-10-19T10:48:28.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac8c0b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac406e380 con 0x7ffac415ddd0 2023-10-19T10:48:29.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac8c0b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac406cef0 con 0x7ffac406cab0 2023-10-19T10:48:29.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffac406a810 0x7ffac40add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffac406a810 0x7ffac40add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34992/0 (socket says 172.21.15.50:34992) 2023-10-19T10:48:29.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 -- 172.21.15.50:0/3575707617 learned_addr learned my addr 172.21.15.50:0/3575707617 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:48:29.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffabbfff640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac406abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 -- 172.21.15.50:0/3575707617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 msgr2=0x7ffac406abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:48:29.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac2d9d640 1 --2- 172.21.15.50:0/3575707617 >> v2:172.21.15.183:3300/0 conn(0x7ffac415ddd0 0x7ffac415e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac406abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 -- 172.21.15.50:0/3575707617 >> v2:172.21.15.183:3300/0 conn(0x7ffac415ddd0 msgr2=0x7ffac415e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:48:29.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 --2- 172.21.15.50:0/3575707617 >> v2:172.21.15.183:3300/0 conn(0x7ffac415ddd0 0x7ffac415e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac259c640 1 -- 172.21.15.50:0/3575707617 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40cc5b0 con 0x7ffac406a810 2023-10-19T10:48:29.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffabbfff640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:48:29.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.995+0000 7ffac2d9d640 1 --2- 172.21.15.50:0/3575707617 >> v2:172.21.15.183:3300/0 conn(0x7ffac415ddd0 0x7ffac415e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:48:29.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.996+0000 7ffac259c640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac406a810 0x7ffac40add10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ffac4071ed0 tx=0x7ffab402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f57b45cb0a67e28 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:48:29.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.996+0000 7ffac1d9b640 1 -- 172.21.15.50:0/3575707617 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab40317d0 con 0x7ffac406a810 2023-10-19T10:48:29.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.996+0000 7ffac1d9b640 1 -- 172.21.15.50:0/3575707617 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4031930 con 0x7ffac406a810 2023-10-19T10:48:29.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.996+0000 7ffac1d9b640 1 -- 172.21.15.50:0/3575707617 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4031cc0 con 0x7ffac406a810 2023-10-19T10:48:29.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 -- 172.21.15.50:0/3575707617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac406a810 msgr2=0x7ffac40add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:29.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac406a810 0x7ffac40add10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ffac4071ed0 tx=0x7ffab402b870 comp rx=0 tx=0).stop 2023-10-19T10:48:29.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 -- 172.21.15.50:0/3575707617 shutdown_connections 2023-10-19T10:48:29.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 --2- 172.21.15.50:0/3575707617 >> v2:172.21.15.183:3300/0 conn(0x7ffac415ddd0 0x7ffac415e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 --2- 172.21.15.50:0/3575707617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac406a810 0x7ffac40add10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 -- 172.21.15.50:0/3575707617 >> 172.21.15.50:0/3575707617 conn(0x7ffac406fe20 msgr2=0x7ffac4070200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:48:29.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 -- 172.21.15.50:0/3575707617 shutdown_connections 2023-10-19T10:48:29.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.997+0000 7ffac8c0b640 1 -- 172.21.15.50:0/3575707617 wait complete. 2023-10-19T10:48:29.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.999+0000 7ffac8c0b640 1 Processor -- start 2023-10-19T10:48:29.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:28.999+0000 7ffac8c0b640 1 -- start start 2023-10-19T10:48:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.000+0000 7ffac8c0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac406a810 0x7ffac4157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.000+0000 7ffac8c0b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac40b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.000+0000 7ffac8c0b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac415ddd0 0x7ffac40b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.000+0000 7ffac8c0b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c5670 con 0x7ffac415ddd0 2023-10-19T10:48:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.000+0000 7ffac8c0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c6430 con 0x7ffac406a810 2023-10-19T10:48:29.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.000+0000 7ffac8c0b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c6710 con 0x7ffac406cab0 2023-10-19T10:48:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac415ddd0 0x7ffac40b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac415ddd0 0x7ffac40b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34994/0 (socket says 172.21.15.50:34994) 2023-10-19T10:48:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 -- 172.21.15.50:0/1094215937 learned_addr learned my addr 172.21.15.50:0/1094215937 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:48:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac259c640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac406a810 0x7ffac4157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffabbfff640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac40b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 -- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 msgr2=0x7ffac40b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffac406cab0 0x7ffac40b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 -- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac406a810 msgr2=0x7ffac4157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffac406a810 0x7ffac4157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 -- 172.21.15.50:0/1094215937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40094d0 con 0x7ffac415ddd0 2023-10-19T10:48:29.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac2d9d640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac415ddd0 0x7ffac40b2980 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00cef0 tx=0x7ffabc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:48:29.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffabaffd640 1 -- 172.21.15.50:0/1094215937 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00d840 con 0x7ffac415ddd0 2023-10-19T10:48:29.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffabaffd640 1 -- 172.21.15.50:0/1094215937 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc00d9c0 con 0x7ffac415ddd0 2023-10-19T10:48:29.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffabaffd640 1 -- 172.21.15.50:0/1094215937 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00dd70 con 0x7ffac415ddd0 2023-10-19T10:48:29.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac8c0b640 1 -- 172.21.15.50:0/1094215937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac40c69f0 con 0x7ffac415ddd0 2023-10-19T10:48:29.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.001+0000 7ffac8c0b640 1 -- 172.21.15.50:0/1094215937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac4158400 con 0x7ffac415ddd0 2023-10-19T10:48:29.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.002+0000 7ffac8c0b640 1 -- 172.21.15.50:0/1094215937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ffac4066e80 con 0x7ffac415ddd0 2023-10-19T10:48:29.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.003+0000 7ffabaffd640 1 -- 172.21.15.50:0/1094215937 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffabc01e3b0 con 0x7ffac415ddd0 2023-10-19T10:48:29.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.003+0000 7ffabaffd640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffaa406a050 0x7ffaa406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:48:29.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.003+0000 7ffabaffd640 1 -- 172.21.15.50:0/1094215937 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ffabc089100 con 0x7ffac415ddd0 2023-10-19T10:48:29.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.003+0000 7ffabaffd640 1 -- 172.21.15.50:0/1094215937 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7ffabc090070 con 0x7ffac415ddd0 2023-10-19T10:48:29.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.004+0000 7ffac259c640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffaa406a050 0x7ffaa406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:48:29.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.004+0000 7ffac8c0b640 1 -- 172.21.15.50:0/1094215937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ffac40cdfe0 con 0x7ffac415ddd0 2023-10-19T10:48:29.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.004+0000 7ffac259c640 1 --2- 172.21.15.50:0/1094215937 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ffaa406a050 0x7ffaa406c510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffab4036000 tx=0x7ffab403d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:48:29.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.195+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb350070960 con 0x7fb3500cc550 2023-10-19T10:48:29.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.196+0000 7fb33effd640 1 -- 172.21.15.50:0/134675549 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fb348079e70 con 0x7fb3500cc550 2023-10-19T10:48:29.199 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:48:29.200 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:48:29.200 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:48:29.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb33006a050 msgr2=0x7fb33006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:29.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb33006a050 0x7fb33006c510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb3400079a0 tx=0x7fb340008040 comp rx=0 tx=0).stop 2023-10-19T10:48:29.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 msgr2=0x7fb35015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:48:29.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 0x7fb35015e000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb348009f40 tx=0x7fb348004a40 comp rx=0 tx=0).stop 2023-10-19T10:48:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 shutdown_connections 2023-10-19T10:48:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb33006a050 0x7fb33006c510 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3500cc550 0x7fb35015e000 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3500ab8d0 0x7fb35014f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 --2- 172.21.15.50:0/134675549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb35006fea0 0x7fb3500ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:48:29.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 >> 172.21.15.50:0/134675549 conn(0x7fb3500ce980 msgr2=0x7fb350157750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:48:29.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 shutdown_connections 2023-10-19T10:48:29.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:48:29.198+0000 7fb3548b1640 1 -- 172.21.15.50:0/134675549 wait complete. 2023-10-19T10:48:29.212 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:48:29.336 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":588.87004353999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.25651406796094411,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}}] 2023-10-19T10:48:29.337 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2565140679609441 }, "state": "open", "uptime": 588.87004354 } ] 2023-10-19T10:48:29.438 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:48:58.274 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:48:58.278 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:49:28.342 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:49:28.346 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:49:29.451 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:49:29.452 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-19T10:49:30.453 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:49:30.457 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:49:30.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.572+0000 7fb2d0360640 1 Processor -- start 2023-10-19T10:49:30.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.572+0000 7fb2d0360640 1 -- start start 2023-10-19T10:49:30.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2d0360640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c80b1f30 0x7fb2c80b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2d0360640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb2c8160340 0x7fb2c81607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2d0360640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c815ee00 con 0x7fb2c806f270 2023-10-19T10:49:30.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2d0360640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c815d6e0 con 0x7fb2c8160340 2023-10-19T10:49:30.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2d0360640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80b2850 con 0x7fb2c80b1f30 2023-10-19T10:49:30.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2cd8fb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb2c8160340 0x7fb2c81607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2cd8fb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb2c8160340 0x7fb2c81607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50154/0 (socket says 172.21.15.50:50154) 2023-10-19T10:49:30.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.573+0000 7fb2cd8fb640 1 -- 172.21.15.50:0/1240121626 learned_addr learned my addr 172.21.15.50:0/1240121626 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:49:30.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.574+0000 7fb2ce0fc640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c80b1f30 0x7fb2c80b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.574+0000 7fb2cd8fb640 1 -- 172.21.15.50:0/1240121626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c80b1f30 msgr2=0x7fb2c80b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:49:30.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.574+0000 7fb2cd8fb640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c80b1f30 0x7fb2c80b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.574+0000 7fb2cd8fb640 1 -- 172.21.15.50:0/1240121626 >> v1:172.21.15.183:6789/0 conn(0x7fb2c806f270 legacy=0x7fb2c806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.574+0000 7fb2cd8fb640 1 -- 172.21.15.50:0/1240121626 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c815f030 con 0x7fb2c8160340 2023-10-19T10:49:30.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.574+0000 7fb2ce0fc640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c80b1f30 0x7fb2c80b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:49:30.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.575+0000 7fb2cd8fb640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c81607a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009870 tx=0x7fb2c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a4c7fd3ff9400d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:49:30.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.575+0000 7fb2cd0fa640 1 -- 172.21.15.50:0/1240121626 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4031640 con 0x7fb2c8160340 2023-10-19T10:49:30.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.575+0000 7fb2cd0fa640 1 -- 172.21.15.50:0/1240121626 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c40317a0 con 0x7fb2c8160340 2023-10-19T10:49:30.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.575+0000 7fb2cd0fa640 1 -- 172.21.15.50:0/1240121626 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4031b30 con 0x7fb2c8160340 2023-10-19T10:49:30.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.575+0000 7fb2d0360640 1 -- 172.21.15.50:0/1240121626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 msgr2=0x7fb2c81607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.575+0000 7fb2d0360640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c81607a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009870 tx=0x7fb2c402b8f0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.585 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.576+0000 7fb2d0360640 1 -- 172.21.15.50:0/1240121626 shutdown_connections 2023-10-19T10:49:30.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.576+0000 7fb2d0360640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c81607a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.576+0000 7fb2d0360640 1 --2- 172.21.15.50:0/1240121626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c80b1f30 0x7fb2c80b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.576+0000 7fb2d0360640 1 -- 172.21.15.50:0/1240121626 >> 172.21.15.50:0/1240121626 conn(0x7fb2c80ccf30 msgr2=0x7fb2c80cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:49:30.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.576+0000 7fb2d0360640 1 -- 172.21.15.50:0/1240121626 shutdown_connections 2023-10-19T10:49:30.586 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.576+0000 7fb2d0360640 1 -- 172.21.15.50:0/1240121626 wait complete. 2023-10-19T10:49:30.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.577+0000 7fb2d0360640 1 Processor -- start 2023-10-19T10:49:30.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.577+0000 7fb2d0360640 1 -- start start 2023-10-19T10:49:30.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2d0360640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c806f270 0x7fb2c8157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2d0360640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c80b1f30 0x7fb2c80b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.587 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2d0360640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c806f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2d0360640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81f2bf0 con 0x7fb2c8160340 2023-10-19T10:49:30.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2d0360640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80c97d0 con 0x7fb2c80b1f30 2023-10-19T10:49:30.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2d0360640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81583d0 con 0x7fb2c806f270 2023-10-19T10:49:30.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.588 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c806f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50158/0 (socket says 172.21.15.50:50158) 2023-10-19T10:49:30.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 -- 172.21.15.50:0/3093833878 learned_addr learned my addr 172.21.15.50:0/3093833878 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:49:30.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce0fc640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c806f270 0x7fb2c8157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2cd8fb640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c80b1f30 0x7fb2c80b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 -- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c806f270 msgr2=0x7fb2c8157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.589 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c806f270 0x7fb2c8157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 -- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c80b1f30 msgr2=0x7fb2c80b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c80b1f30 0x7fb2c80b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 -- 172.21.15.50:0/3093833878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40094d0 con 0x7fb2c8160340 2023-10-19T10:49:30.590 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2ce8fd640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c806f700 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c6b0 tx=0x7fb2c000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:49:30.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0003b70 con 0x7fb2c8160340 2023-10-19T10:49:30.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.578+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0003cf0 con 0x7fb2c8160340 2023-10-19T10:49:30.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.579+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c80c8e70 con 0x7fb2c8160340 2023-10-19T10:49:30.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.579+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0003ea0 con 0x7fb2c8160340 2023-10-19T10:49:30.591 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.579+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c80c9110 con 0x7fb2c8160340 2023-10-19T10:49:30.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.579+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c80c9910 con 0x7fb2c8160340 2023-10-19T10:49:30.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.580+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb2c0005310 con 0x7fb2c8160340 2023-10-19T10:49:30.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.580+0000 7fb2baffd640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb2ac06a050 0x7fb2ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.581+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb2c00882f0 con 0x7fb2c8160340 2023-10-19T10:49:30.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.581+0000 7fb2ce0fc640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb2ac06a050 0x7fb2ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.592 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.581+0000 7fb2ce0fc640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb2ac06a050 0x7fb2ac06c510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2c815a780 tx=0x7fb2bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:49:30.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.585+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb2c80c9910 con 0x7fb2c8160340 2023-10-19T10:49:30.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.596+0000 7ff2f7518640 1 Processor -- start 2023-10-19T10:49:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.596+0000 7ff2f7518640 1 -- start start 2023-10-19T10:49:30.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f7518640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f7518640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2f00d1040 0x7ff2f00b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f7518640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f00cde90 con 0x7ff2f00cd960 2023-10-19T10:49:30.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f7518640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f00ce170 con 0x7ff2f00d1040 2023-10-19T10:49:30.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f7518640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00b1c80 con 0x7ff2f015c680 2023-10-19T10:49:30.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f52b4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f5ab5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2f00d1040 0x7ff2f00b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f52b4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:35804/0 (socket says 172.21.15.50:35804) 2023-10-19T10:49:30.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f52b4640 1 -- 172.21.15.50:0/1454648757 learned_addr learned my addr 172.21.15.50:0/1454648757 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:49:30.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f5ab5640 1 -- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 msgr2=0x7ff2f00d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:49:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f5ab5640 1 --2- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.597+0000 7ff2f5ab5640 1 -- 172.21.15.50:0/1454648757 >> v1:172.21.15.183:6789/0 conn(0x7ff2f00cd960 legacy=0x7ff2f00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.598+0000 7ff2f5ab5640 1 -- 172.21.15.50:0/1454648757 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00b1f60 con 0x7ff2f00d1040 2023-10-19T10:49:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.598+0000 7ff2f5ab5640 1 --2- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00b2760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009d90 tx=0x7ff2e802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ccd03329c0a7d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:49:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2e7fff640 1 -- 172.21.15.50:0/1454648757 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e802cce0 con 0x7ff2f00d1040 2023-10-19T10:49:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2e7fff640 1 -- 172.21.15.50:0/1454648757 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e802ce40 con 0x7ff2f00d1040 2023-10-19T10:49:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2e7fff640 1 -- 172.21.15.50:0/1454648757 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8034660 con 0x7ff2f00d1040 2023-10-19T10:49:30.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 -- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 msgr2=0x7ff2f00b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 --2- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00b2760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009d90 tx=0x7ff2e802bf30 comp rx=0 tx=0).stop 2023-10-19T10:49:30.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 -- 172.21.15.50:0/1454648757 shutdown_connections 2023-10-19T10:49:30.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 --2- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00b2760 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 --2- 172.21.15.50:0/1454648757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 -- 172.21.15.50:0/1454648757 >> 172.21.15.50:0/1454648757 conn(0x7ff2f00d2370 msgr2=0x7ff2f00d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:49:30.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 -- 172.21.15.50:0/1454648757 shutdown_connections 2023-10-19T10:49:30.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.599+0000 7ff2f7518640 1 -- 172.21.15.50:0/1454648757 wait complete. 2023-10-19T10:49:30.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 Processor -- start 2023-10-19T10:49:30.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 -- start start 2023-10-19T10:49:30.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2f00cd960 0x7ff2f0152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00b62f0 con 0x7ff2f00cd960 2023-10-19T10:49:30.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00b4df0 con 0x7ff2f00d1040 2023-10-19T10:49:30.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f7518640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00b4550 con 0x7ff2f015c680 2023-10-19T10:49:30.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f52b4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2f00cd960 0x7ff2f0152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f52b4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2f00cd960 0x7ff2f0152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50168/0 (socket says 172.21.15.50:50168) 2023-10-19T10:49:30.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.601+0000 7ff2f52b4640 1 -- 172.21.15.50:0/3449720694 learned_addr learned my addr 172.21.15.50:0/3449720694 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:49:30.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f5ab5640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f4ab3640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f52b4640 1 -- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 msgr2=0x7ff2f00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f52b4640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2f015c680 0x7ff2f00ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f52b4640 1 -- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 msgr2=0x7ff2f00c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f52b4640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f52b4640 1 -- 172.21.15.50:0/3449720694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009a70 con 0x7ff2f00cd960 2023-10-19T10:49:30.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f4ab3640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2f00d1040 0x7ff2f00c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:49:30.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f52b4640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2f00cd960 0x7ff2f0152910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e7d0 tx=0x7ff2ec00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:49:30.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec009560 con 0x7ff2f00cd960 2023-10-19T10:49:30.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec00c940 con 0x7ff2f00cd960 2023-10-19T10:49:30.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f7518640 1 -- 172.21.15.50:0/3449720694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f00b4830 con 0x7ff2f00cd960 2023-10-19T10:49:30.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00eee0 con 0x7ff2f00cd960 2023-10-19T10:49:30.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.602+0000 7ff2f7518640 1 -- 172.21.15.50:0/3449720694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f00b6520 con 0x7ff2f00cd960 2023-10-19T10:49:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.603+0000 7ff2f7518640 1 -- 172.21.15.50:0/3449720694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff2f00cfc70 con 0x7ff2f00cd960 2023-10-19T10:49:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.604+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff2ec00ccf0 con 0x7ff2f00cd960 2023-10-19T10:49:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.604+0000 7ff2e5ffb640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:49:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.604+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff2ec0891e0 con 0x7ff2f00cd960 2023-10-19T10:49:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.605+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7ff2ec089570 con 0x7ff2f00cd960 2023-10-19T10:49:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.605+0000 7ff2f7518640 1 -- 172.21.15.50:0/3449720694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff2f00d05b0 con 0x7ff2f00cd960 2023-10-19T10:49:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.605+0000 7ff2f4ab3640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:49:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.605+0000 7ff2f4ab3640 1 --2- 172.21.15.50:0/3449720694 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff2d406a050 0x7ff2d406c510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff2f015afd0 tx=0x7ff2e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:49:30.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.798+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2c80c9910 con 0x7fb2c8160340 2023-10-19T10:49:30.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.798+0000 7fb2baffd640 1 -- 172.21.15.50:0/3093833878 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fb2c80c9910 con 0x7fb2c8160340 2023-10-19T10:49:30.802 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:49:30.802 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:49:30.803 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:49:30.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb2ac06a050 msgr2=0x7fb2ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb2ac06a050 0x7fb2ac06c510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2c815a780 tx=0x7fb2bc008040 comp rx=0 tx=0).stop 2023-10-19T10:49:30.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 msgr2=0x7fb2c806f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:49:30.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c806f700 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c6b0 tx=0x7fb2c000cb70 comp rx=0 tx=0).stop 2023-10-19T10:49:30.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 shutdown_connections 2023-10-19T10:49:30.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb2ac06a050 0x7fb2ac06c510 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2c8160340 0x7fb2c806f700 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c80b1f30 0x7fb2c80b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 --2- 172.21.15.50:0/3093833878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb2c806f270 0x7fb2c8157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:49:30.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 >> 172.21.15.50:0/3093833878 conn(0x7fb2c80ccf30 msgr2=0x7fb2c80b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:49:30.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 shutdown_connections 2023-10-19T10:49:30.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:49:30.800+0000 7fb2d0360640 1 -- 172.21.15.50:0/3093833878 wait complete. 2023-10-19T10:49:30.813 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:49:30.936 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":650.470449143,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.22248425848947159,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}}] 2023-10-19T10:49:30.936 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2224842584894716 }, "state": "open", "uptime": 650.470449143 } ] 2023-10-19T10:49:31.038 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:49:58.412 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:49:58.416 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:50:00.447 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:00.444+0000 7ff2e5ffb640 1 -- 172.21.15.50:0/3449720694 <== mon.0 v2:172.21.15.50:3300/0 7 ==== log(1 entries from seq 560 at 2023-10-19T10:50:00.000169+0000) v1 ==== 187+0+0 (secure 0 0 0) 0x7ff2ec05c080 con 0x7ff2f00cd960 2023-10-19T10:50:00.447 INFO:teuthology.orchestra.run.smithi050.stdout: cluster: 2023-10-19T10:50:00.448 INFO:teuthology.orchestra.run.smithi050.stdout: id: 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 2023-10-19T10:50:00.448 INFO:teuthology.orchestra.run.smithi050.stdout: health: HEALTH_OK 2023-10-19T10:50:00.448 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:50:00.448 INFO:teuthology.orchestra.run.smithi050.stdout: services: 2023-10-19T10:50:00.448 INFO:teuthology.orchestra.run.smithi050.stdout: mon: 3 daemons, quorum a,b,c (age 13m) 2023-10-19T10:50:00.449 INFO:teuthology.orchestra.run.smithi050.stdout: mgr: x(active, since 12m), standbys: y 2023-10-19T10:50:00.449 INFO:teuthology.orchestra.run.smithi050.stdout: mds: 1/1 daemons up, 3 standby 2023-10-19T10:50:00.449 INFO:teuthology.orchestra.run.smithi050.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 12m) 2023-10-19T10:50:00.449 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:50:00.449 INFO:teuthology.orchestra.run.smithi050.stdout: data: 2023-10-19T10:50:00.450 INFO:teuthology.orchestra.run.smithi050.stdout: volumes: 1/1 healthy 2023-10-19T10:50:00.450 INFO:teuthology.orchestra.run.smithi050.stdout: pools: 4 pools, 105 pgs 2023-10-19T10:50:00.450 INFO:teuthology.orchestra.run.smithi050.stdout: objects: 24 objects, 581 KiB 2023-10-19T10:50:00.450 INFO:teuthology.orchestra.run.smithi050.stdout: usage: 234 MiB used, 720 GiB / 720 GiB avail 2023-10-19T10:50:00.450 INFO:teuthology.orchestra.run.smithi050.stdout: pgs: 105 active+clean 2023-10-19T10:50:00.451 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:50:00.451 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:50:00.451 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:50:00.000169+0000 mon.a [INF] overall HEALTH_OK 2023-10-19T10:50:28.501 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:50:28.505 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:50:31.050 ERROR:tasks.ceph_test_case:Log output: cluster: id: 2b47c9ed-f967-49ff-9d46-d95ee8c39a83 health: HEALTH_OK services: mon: 3 daemons, quorum a,b,c (age 13m) mgr: x(active, since 12m), standbys: y mds: 1/1 daemons up, 3 standby osd: 8 osds: 8 up (since 3m), 8 in (since 12m) data: volumes: 1/1 healthy pools: 4 pools, 105 pgs objects: 24 objects, 581 KiB usage: 234 MiB used, 720 GiB / 720 GiB avail pgs: 105 active+clean 2023-10-19T10:50:00.000169+0000 mon.a [INF] overall HEALTH_OK 2023-10-19T10:50:31.051 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-19T10:50:32.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:50:32.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:50:32.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.171+0000 7fbb856ab640 1 Processor -- start 2023-10-19T10:50:32.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.171+0000 7fbb856ab640 1 -- start start 2023-10-19T10:50:32.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb856ab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbb8015c570 0x7fbb800d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb856ab640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbb800cd920 0x7fbb800d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb856ab640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8006aa30 con 0x7fbb8015c950 2023-10-19T10:50:32.176 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb856ab640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8006b2d0 con 0x7fbb800cd920 2023-10-19T10:50:32.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb856ab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800ce090 con 0x7fbb8015c570 2023-10-19T10:50:32.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb7f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbb800cd920 0x7fbb800d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb7f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbb800cd920 0x7fbb800d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42342/0 (socket says 172.21.15.50:42342) 2023-10-19T10:50:32.177 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb7f7fe640 1 -- 172.21.15.50:0/3081665924 learned_addr learned my addr 172.21.15.50:0/3081665924 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:50:32.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.172+0000 7fbb7effd640 1 --2- 172.21.15.50:0/3081665924 >> v2:172.21.15.183:3300/0 conn(0x7fbb8015c570 0x7fbb800d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.173+0000 7fbb7f7fe640 1 -- 172.21.15.50:0/3081665924 >> v2:172.21.15.183:3300/0 conn(0x7fbb8015c570 msgr2=0x7fbb800d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:50:32.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.173+0000 7fbb7f7fe640 1 --2- 172.21.15.50:0/3081665924 >> v2:172.21.15.183:3300/0 conn(0x7fbb8015c570 0x7fbb800d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.173+0000 7fbb7f7fe640 1 -- 172.21.15.50:0/3081665924 >> v1:172.21.15.183:6789/0 conn(0x7fbb8015c950 legacy=0x7fbb8006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:50:32.178 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.173+0000 7fbb7f7fe640 1 -- 172.21.15.50:0/3081665924 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8006ac80 con 0x7fbb800cd920 2023-10-19T10:50:32.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.174+0000 7fbb7f7fe640 1 --2- 172.21.15.50:0/3081665924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb800d26a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbb800d2670 tx=0x7fbb7802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cce26157115ba91 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:50:32.181 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.174+0000 7fbb7dffb640 1 -- 172.21.15.50:0/3081665924 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7802c9f0 con 0x7fbb800cd920 2023-10-19T10:50:32.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.174+0000 7fbb7dffb640 1 -- 172.21.15.50:0/3081665924 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7802cb50 con 0x7fbb800cd920 2023-10-19T10:50:32.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.174+0000 7fbb7dffb640 1 -- 172.21.15.50:0/3081665924 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb780343f0 con 0x7fbb800cd920 2023-10-19T10:50:32.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 -- 172.21.15.50:0/3081665924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 msgr2=0x7fbb800d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 --2- 172.21.15.50:0/3081665924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb800d26a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbb800d2670 tx=0x7fbb7802be50 comp rx=0 tx=0).stop 2023-10-19T10:50:32.182 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 -- 172.21.15.50:0/3081665924 shutdown_connections 2023-10-19T10:50:32.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 --2- 172.21.15.50:0/3081665924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb800d26a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 --2- 172.21.15.50:0/3081665924 >> v2:172.21.15.183:3300/0 conn(0x7fbb8015c570 0x7fbb800d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 -- 172.21.15.50:0/3081665924 >> 172.21.15.50:0/3081665924 conn(0x7fbb80089ff0 msgr2=0x7fbb8009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:50:32.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 -- 172.21.15.50:0/3081665924 shutdown_connections 2023-10-19T10:50:32.183 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.175+0000 7fbb856ab640 1 -- 172.21.15.50:0/3081665924 wait complete. 2023-10-19T10:50:32.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.176+0000 7fbb856ab640 1 Processor -- start 2023-10-19T10:50:32.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.176+0000 7fbb856ab640 1 -- start start 2023-10-19T10:50:32.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb856ab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb80158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb856ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb8015c570 0x7fbb800c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb856ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb8015c950 0x7fbb80158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.184 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb856ab640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80159fd0 con 0x7fbb800cd920 2023-10-19T10:50:32.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb856ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800c7c50 con 0x7fbb8015c570 2023-10-19T10:50:32.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb856ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800c73d0 con 0x7fbb8015c950 2023-10-19T10:50:32.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb80158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb80158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42344/0 (socket says 172.21.15.50:42344) 2023-10-19T10:50:32.185 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 -- 172.21.15.50:0/2865196117 learned_addr learned my addr 172.21.15.50:0/2865196117 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:50:32.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7e7fc640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb8015c570 0x7fbb800c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7f7fe640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb8015c950 0x7fbb80158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 -- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb8015c950 msgr2=0x7fbb80158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb8015c950 0x7fbb80158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 -- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb8015c570 msgr2=0x7fbb800c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb8015c570 0x7fbb800c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7effd640 1 -- 172.21.15.50:0/2865196117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78009b30 con 0x7fbb800cd920 2023-10-19T10:50:32.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7e7fc640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb8015c570 0x7fbb800c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:50:32.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.177+0000 7fbb7f7fe640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb8015c950 0x7fbb80158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:50:32.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.178+0000 7fbb7effd640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb80158120 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000d800 tx=0x7fbb7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:50:32.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.178+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7000b6c0 con 0x7fbb800cd920 2023-10-19T10:50:32.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.178+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7000b820 con 0x7fbb800cd920 2023-10-19T10:50:32.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.178+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb800c7690 con 0x7fbb800cd920 2023-10-19T10:50:32.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.178+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7000bbd0 con 0x7fbb800cd920 2023-10-19T10:50:32.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.178+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8015d040 con 0x7fbb800cd920 2023-10-19T10:50:32.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.179+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8015cd70 con 0x7fbb800cd920 2023-10-19T10:50:32.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.180+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbb7000bd30 con 0x7fbb800cd920 2023-10-19T10:50:32.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.180+0000 7fbb777fe640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbb6006a050 0x7fbb6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.180+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fbb70088b90 con 0x7fbb800cd920 2023-10-19T10:50:32.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.180+0000 7fbb7e7fc640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbb6006a050 0x7fbb6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.181+0000 7fbb7e7fc640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbb6006a050 0x7fbb6006c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbb800cf6c0 tx=0x7fbb68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:50:32.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.188+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbb8015cd70 con 0x7fbb800cd920 2023-10-19T10:50:32.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.200+0000 7f3e31bbd640 1 Processor -- start 2023-10-19T10:50:32.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.200+0000 7f3e31bbd640 1 -- start start 2023-10-19T10:50:32.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e31bbd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3e2c15c680 0x7f3e2c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e31bbd640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2c0cde90 con 0x7f3e2c0d1040 2023-10-19T10:50:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e31bbd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2c0ce170 con 0x7f3e2c0cd960 2023-10-19T10:50:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e31bbd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2c0b1c80 con 0x7f3e2c15c680 2023-10-19T10:50:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3e2c15c680 0x7f3e2c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2b7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3e2c15c680 0x7f3e2c0d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42360/0 (socket says 172.21.15.50:42360) 2023-10-19T10:50:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2b7fe640 1 -- 172.21.15.50:0/510358848 learned_addr learned my addr 172.21.15.50:0/510358848 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:50:32.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2b7fe640 1 -- 172.21.15.50:0/510358848 >> v1:172.21.15.183:6789/0 conn(0x7f3e2c0cd960 legacy=0x7f3e2c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2b7fe640 1 -- 172.21.15.50:0/510358848 >> v1:172.21.15.50:6789/0 conn(0x7f3e2c0d1040 legacy=0x7f3e2c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2a7fc640 1 -- 172.21.15.50:0/510358848 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664450477 0 0) 0x7f3e2c0cde90 con 0x7f3e2c0d1040 2023-10-19T10:50:32.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.201+0000 7f3e2b7fe640 1 -- 172.21.15.50:0/510358848 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0b1f60 con 0x7f3e2c15c680 2023-10-19T10:50:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.202+0000 7f3e2b7fe640 1 --2- 172.21.15.50:0/510358848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c15c680 0x7f3e2c0d0090 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009850 tx=0x7f3e1c02baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f3cf5b33973e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:50:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.202+0000 7f3e2a7fc640 1 -- 172.21.15.50:0/510358848 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c004230 con 0x7f3e2c15c680 2023-10-19T10:50:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.202+0000 7f3e2a7fc640 1 -- 172.21.15.50:0/510358848 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c004390 con 0x7f3e2c15c680 2023-10-19T10:50:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.202+0000 7f3e2a7fc640 1 -- 172.21.15.50:0/510358848 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c02d860 con 0x7f3e2c15c680 2023-10-19T10:50:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 -- 172.21.15.50:0/510358848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c15c680 msgr2=0x7f3e2c0d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 --2- 172.21.15.50:0/510358848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c15c680 0x7f3e2c0d0090 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009850 tx=0x7f3e1c02baa0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 -- 172.21.15.50:0/510358848 shutdown_connections 2023-10-19T10:50:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 --2- 172.21.15.50:0/510358848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c15c680 0x7f3e2c0d0090 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 -- 172.21.15.50:0/510358848 >> 172.21.15.50:0/510358848 conn(0x7f3e2c0d2370 msgr2=0x7f3e2c0d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:50:32.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 -- 172.21.15.50:0/510358848 shutdown_connections 2023-10-19T10:50:32.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.203+0000 7f3e31bbd640 1 -- 172.21.15.50:0/510358848 wait complete. 2023-10-19T10:50:32.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.204+0000 7f3e31bbd640 1 Processor -- start 2023-10-19T10:50:32.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.204+0000 7f3e31bbd640 1 -- start start 2023-10-19T10:50:32.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e31bbd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c0cd960 0x7f3e2c14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e31bbd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0d1040 0x7f3e2c0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e31bbd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2c15c680 0x7f3e2c06aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e31bbd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cba20 con 0x7f3e2c0d1040 2023-10-19T10:50:32.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e31bbd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0ca8a0 con 0x7f3e2c0cd960 2023-10-19T10:50:32.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e31bbd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c14fae0 con 0x7f3e2c15c680 2023-10-19T10:50:32.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e2affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0d1040 0x7f3e2c0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e2affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0d1040 0x7f3e2c0c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42364/0 (socket says 172.21.15.50:42364) 2023-10-19T10:50:32.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e2affd640 1 -- 172.21.15.50:0/4247031364 learned_addr learned my addr 172.21.15.50:0/4247031364 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:50:32.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e2b7fe640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c0cd960 0x7f3e2c14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.205+0000 7f3e2bfff640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2c15c680 0x7f3e2c06aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2affd640 1 -- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2c15c680 msgr2=0x7f3e2c06aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2affd640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2c15c680 0x7f3e2c06aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2affd640 1 -- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c0cd960 msgr2=0x7f3e2c14f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2affd640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c0cd960 0x7f3e2c14f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2affd640 1 -- 172.21.15.50:0/4247031364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c009460 con 0x7f3e2c0d1040 2023-10-19T10:50:32.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2b7fe640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c0cd960 0x7f3e2c14f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:50:32.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2bfff640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2c15c680 0x7f3e2c06aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:50:32.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e2affd640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e2c0d1040 0x7f3e2c0c9440 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000c400 tx=0x7f3e2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:50:32.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e13fff640 1 -- 172.21.15.50:0/4247031364 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e200052d0 con 0x7f3e2c0d1040 2023-10-19T10:50:32.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e13fff640 1 -- 172.21.15.50:0/4247031364 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e20005430 con 0x7f3e2c0d1040 2023-10-19T10:50:32.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e31bbd640 1 -- 172.21.15.50:0/4247031364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2c0c9c90 con 0x7f3e2c0d1040 2023-10-19T10:50:32.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e13fff640 1 -- 172.21.15.50:0/4247031364 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20003830 con 0x7f3e2c0d1040 2023-10-19T10:50:32.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.206+0000 7f3e31bbd640 1 -- 172.21.15.50:0/4247031364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c0ca250 con 0x7f3e2c0d1040 2023-10-19T10:50:32.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.207+0000 7f3e31bbd640 1 -- 172.21.15.50:0/4247031364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3e2c0caa30 con 0x7f3e2c0d1040 2023-10-19T10:50:32.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.208+0000 7f3e13fff640 1 -- 172.21.15.50:0/4247031364 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e20003990 con 0x7f3e2c0d1040 2023-10-19T10:50:32.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.208+0000 7f3e13fff640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e1806a080 0x7f3e1806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:50:32.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.208+0000 7f3e13fff640 1 -- 172.21.15.50:0/4247031364 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3e200883b0 con 0x7f3e2c0d1040 2023-10-19T10:50:32.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.209+0000 7f3e13fff640 1 -- 172.21.15.50:0/4247031364 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f3e20088820 con 0x7f3e2c0d1040 2023-10-19T10:50:32.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.209+0000 7f3e2b7fe640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e1806a080 0x7f3e1806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:50:32.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.209+0000 7f3e31bbd640 1 -- 172.21.15.50:0/4247031364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f3e2c066e80 con 0x7f3e2c0d1040 2023-10-19T10:50:32.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.209+0000 7f3e2b7fe640 1 --2- 172.21.15.50:0/4247031364 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3e1806a080 0x7f3e1806c540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c035000 tx=0x7f3e1c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:50:32.403 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.401+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb8015cd70 con 0x7fbb800cd920 2023-10-19T10:50:32.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.401+0000 7fbb777fe640 1 -- 172.21.15.50:0/2865196117 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fbb8015cd70 con 0x7fbb800cd920 2023-10-19T10:50:32.405 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:50:32.405 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:50:32.405 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:50:32.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbb6006a050 msgr2=0x7fbb6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbb6006a050 0x7fbb6006c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbb800cf6c0 tx=0x7fbb68009040 comp rx=0 tx=0).stop 2023-10-19T10:50:32.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 msgr2=0x7fbb80158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:50:32.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb80158120 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000d800 tx=0x7fbb7000dcc0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 shutdown_connections 2023-10-19T10:50:32.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbb6006a050 0x7fbb6006c510 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb8015c950 0x7fbb80158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb8015c570 0x7fbb800c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 --2- 172.21.15.50:0/2865196117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb800cd920 0x7fbb80158120 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:50:32.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 >> 172.21.15.50:0/2865196117 conn(0x7fbb80089ff0 msgr2=0x7fbb8015bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:50:32.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 shutdown_connections 2023-10-19T10:50:32.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:50:32.403+0000 7fbb856ab640 1 -- 172.21.15.50:0/2865196117 wait complete. 2023-10-19T10:50:32.417 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:50:32.540 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":712.07384937799998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.19296759503901068,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}}] 2023-10-19T10:50:32.541 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.19296759503901068 }, "state": "open", "uptime": 712.073849378 } ] 2023-10-19T10:50:32.642 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:50:58.606 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:50:58.609 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:51:28.665 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:51:28.669 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:51:32.655 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:51:32.656 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-19T10:51:33.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-19T10:51:33.661 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:51:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.777+0000 7fc257a0b640 1 Processor -- start 2023-10-19T10:51:33.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.777+0000 7fc257a0b640 1 -- start start 2023-10-19T10:51:33.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc257a0b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc2500d2910 0x7fc2500b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc257a0b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc2500d0fc0 0x7fc2500ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc257a0b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500b1910 con 0x7fc2500d2cf0 2023-10-19T10:51:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc257a0b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500cff90 con 0x7fc2500d2910 2023-10-19T10:51:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc257a0b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500ceef0 con 0x7fc2500d0fc0 2023-10-19T10:51:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc2500d2910 0x7fc2500b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc2500d2910 0x7fc2500b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45672/0 (socket says 172.21.15.50:45672) 2023-10-19T10:51:33.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 -- 172.21.15.50:0/77918744 learned_addr learned my addr 172.21.15.50:0/77918744 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:51:33.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc254fa6640 1 --1- 172.21.15.50:0/77918744 >> v1:172.21.15.50:6789/0 conn(0x7fc2500d2cf0 0x7fc2500d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55242/0 (socket says 172.21.15.50:55242) 2023-10-19T10:51:33.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc255fa8640 1 --2- 172.21.15.50:0/77918744 >> v2:172.21.15.183:3300/0 conn(0x7fc2500d0fc0 0x7fc2500ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 -- 172.21.15.50:0/77918744 >> v2:172.21.15.183:3300/0 conn(0x7fc2500d0fc0 msgr2=0x7fc2500ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:51:33.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 --2- 172.21.15.50:0/77918744 >> v2:172.21.15.183:3300/0 conn(0x7fc2500d0fc0 0x7fc2500ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 -- 172.21.15.50:0/77918744 >> v1:172.21.15.50:6789/0 conn(0x7fc2500d2cf0 legacy=0x7fc2500d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc2557a7640 1 -- 172.21.15.50:0/77918744 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500cf1d0 con 0x7fc2500d2910 2023-10-19T10:51:33.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc247fff640 1 -- 172.21.15.50:0/77918744 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692552275 0 0) 0x7fc2500b1910 con 0x7fc2500d2cf0 2023-10-19T10:51:33.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.778+0000 7fc255fa8640 1 --2- 172.21.15.50:0/77918744 >> v2:172.21.15.183:3300/0 conn(0x7fc2500d0fc0 0x7fc2500ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:51:33.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.779+0000 7fc2557a7640 1 --2- 172.21.15.50:0/77918744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2500b1bf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc25006a4f0 tx=0x7fc24c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71174429189c6b19 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:51:33.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.779+0000 7fc247fff640 1 -- 172.21.15.50:0/77918744 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c004230 con 0x7fc2500d2910 2023-10-19T10:51:33.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.779+0000 7fc247fff640 1 -- 172.21.15.50:0/77918744 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c004390 con 0x7fc2500d2910 2023-10-19T10:51:33.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.779+0000 7fc247fff640 1 -- 172.21.15.50:0/77918744 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c030840 con 0x7fc2500d2910 2023-10-19T10:51:33.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 -- 172.21.15.50:0/77918744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 msgr2=0x7fc2500b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 --2- 172.21.15.50:0/77918744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2500b1bf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc25006a4f0 tx=0x7fc24c02b980 comp rx=0 tx=0).stop 2023-10-19T10:51:33.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 -- 172.21.15.50:0/77918744 shutdown_connections 2023-10-19T10:51:33.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 --2- 172.21.15.50:0/77918744 >> v2:172.21.15.183:3300/0 conn(0x7fc2500d0fc0 0x7fc2500ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 --2- 172.21.15.50:0/77918744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2500b1bf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 -- 172.21.15.50:0/77918744 >> 172.21.15.50:0/77918744 conn(0x7fc250089ff0 msgr2=0x7fc25009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:51:33.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 -- 172.21.15.50:0/77918744 shutdown_connections 2023-10-19T10:51:33.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.780+0000 7fc257a0b640 1 -- 172.21.15.50:0/77918744 wait complete. 2023-10-19T10:51:33.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.781+0000 7fc257a0b640 1 Processor -- start 2023-10-19T10:51:33.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.781+0000 7fc257a0b640 1 -- start start 2023-10-19T10:51:33.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc257a0b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2500d0fc0 0x7fc250155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc257a0b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2501e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc257a0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2500d2cf0 0x7fc2501ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc257a0b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500cb1b0 con 0x7fc2500d2910 2023-10-19T10:51:33.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc257a0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc250156440 con 0x7fc2500d2cf0 2023-10-19T10:51:33.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc257a0b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500ca5a0 con 0x7fc2500d0fc0 2023-10-19T10:51:33.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc254fa6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2501e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc254fa6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2501e9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45682/0 (socket says 172.21.15.50:45682) 2023-10-19T10:51:33.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc254fa6640 1 -- 172.21.15.50:0/4052571595 learned_addr learned my addr 172.21.15.50:0/4052571595 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:51:33.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc255fa8640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2500d2cf0 0x7fc2501ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.782+0000 7fc2557a7640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2500d0fc0 0x7fc250155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc254fa6640 1 -- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2500d0fc0 msgr2=0x7fc250155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc254fa6640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2500d0fc0 0x7fc250155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc254fa6640 1 -- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2500d2cf0 msgr2=0x7fc2501ea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc254fa6640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2500d2cf0 0x7fc2501ea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc254fa6640 1 -- 172.21.15.50:0/4052571595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c009460 con 0x7fc2500d2910 2023-10-19T10:51:33.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc2557a7640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2500d0fc0 0x7fc250155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:51:33.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc254fa6640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2501e9eb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c430 tx=0x7fc24000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:51:33.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400074e0 con 0x7fc2500d2910 2023-10-19T10:51:33.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc240007660 con 0x7fc2500d2910 2023-10-19T10:51:33.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240004a20 con 0x7fc2500d2910 2023-10-19T10:51:33.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2500ca880 con 0x7fc2500d2910 2023-10-19T10:51:33.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.783+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2500cae40 con 0x7fc2500d2910 2023-10-19T10:51:33.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.785+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250073990 con 0x7fc2500d2910 2023-10-19T10:51:33.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.785+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc240004b80 con 0x7fc2500d2910 2023-10-19T10:51:33.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.786+0000 7fc245ffb640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc23406a050 0x7fc23406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.786+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fc240088580 con 0x7fc2500d2910 2023-10-19T10:51:33.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.786+0000 7fc2557a7640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc23406a050 0x7fc23406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.786+0000 7fc2557a7640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc23406a050 0x7fc23406c510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc24c035000 tx=0x7fc24c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:51:33.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.790+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc240053a40 con 0x7fc2500d2910 2023-10-19T10:51:33.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.801+0000 7ff25a71f640 1 Processor -- start 2023-10-19T10:51:33.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.801+0000 7ff25a71f640 1 -- start start 2023-10-19T10:51:33.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff25a71f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff25415c680 0x7ff2540d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff25a71f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540cd960 0x7ff2540d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff25a71f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2540d0e70 0x7ff2540ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff25a71f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540d1300 con 0x7ff2540d0e70 2023-10-19T10:51:33.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff25a71f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540cde90 con 0x7ff25415c680 2023-10-19T10:51:33.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff25a71f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540ce170 con 0x7ff2540cd960 2023-10-19T10:51:33.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff258cbc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2540d0e70 0x7ff2540ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff258cbc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff2540d0e70 0x7ff2540ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45684/0 (socket says 172.21.15.50:45684) 2023-10-19T10:51:33.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff258cbc640 1 -- 172.21.15.50:0/3039172790 learned_addr learned my addr 172.21.15.50:0/3039172790 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:51:33.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff253fff640 1 --2- 172.21.15.50:0/3039172790 >> v2:172.21.15.183:3300/0 conn(0x7ff25415c680 0x7ff2540d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.802+0000 7ff2537fe640 1 --2- 172.21.15.50:0/3039172790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540cd960 0x7ff2540d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.803+0000 7ff258cbc640 1 -- 172.21.15.50:0/3039172790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540cd960 msgr2=0x7ff2540d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:51:33.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.803+0000 7ff258cbc640 1 --2- 172.21.15.50:0/3039172790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540cd960 0x7ff2540d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.803+0000 7ff258cbc640 1 -- 172.21.15.50:0/3039172790 >> v2:172.21.15.183:3300/0 conn(0x7ff25415c680 msgr2=0x7ff2540d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:51:33.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.803+0000 7ff258cbc640 1 --2- 172.21.15.50:0/3039172790 >> v2:172.21.15.183:3300/0 conn(0x7ff25415c680 0x7ff2540d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.803+0000 7ff258cbc640 1 -- 172.21.15.50:0/3039172790 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540b1c80 con 0x7ff2540d0e70 2023-10-19T10:51:33.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.804+0000 7ff258cbc640 1 --2- 172.21.15.50:0/3039172790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540d0e70 0x7ff2540ced40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff25408c4d0 tx=0x7ff24c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c031c52508839b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:51:33.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.804+0000 7ff252ffd640 1 -- 172.21.15.50:0/3039172790 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c02cb00 con 0x7ff2540d0e70 2023-10-19T10:51:33.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.804+0000 7ff252ffd640 1 -- 172.21.15.50:0/3039172790 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c02cc60 con 0x7ff2540d0e70 2023-10-19T10:51:33.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.804+0000 7ff252ffd640 1 -- 172.21.15.50:0/3039172790 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0343f0 con 0x7ff2540d0e70 2023-10-19T10:51:33.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 -- 172.21.15.50:0/3039172790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540d0e70 msgr2=0x7ff2540ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 --2- 172.21.15.50:0/3039172790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540d0e70 0x7ff2540ced40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff25408c4d0 tx=0x7ff24c02bf60 comp rx=0 tx=0).stop 2023-10-19T10:51:33.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 -- 172.21.15.50:0/3039172790 shutdown_connections 2023-10-19T10:51:33.814 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 --2- 172.21.15.50:0/3039172790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540d0e70 0x7ff2540ced40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 --2- 172.21.15.50:0/3039172790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540cd960 0x7ff2540d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 --2- 172.21.15.50:0/3039172790 >> v2:172.21.15.183:3300/0 conn(0x7ff25415c680 0x7ff2540d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 -- 172.21.15.50:0/3039172790 >> 172.21.15.50:0/3039172790 conn(0x7ff2540d2370 msgr2=0x7ff2540d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:51:33.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 -- 172.21.15.50:0/3039172790 shutdown_connections 2023-10-19T10:51:33.815 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.805+0000 7ff25a71f640 1 -- 172.21.15.50:0/3039172790 wait complete. 2023-10-19T10:51:33.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.806+0000 7ff25a71f640 1 Processor -- start 2023-10-19T10:51:33.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.806+0000 7ff25a71f640 1 -- start start 2023-10-19T10:51:33.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff25a71f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540cd960 0x7ff2541f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff25a71f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540d0e70 0x7ff2540c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff25a71f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff25415c680 0x7ff2541f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff25a71f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff254152f50 con 0x7ff2540cd960 2023-10-19T10:51:33.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff25a71f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c88d0 con 0x7ff25415c680 2023-10-19T10:51:33.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff25a71f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c8bb0 con 0x7ff2540d0e70 2023-10-19T10:51:33.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540cd960 0x7ff2541f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540cd960 0x7ff2541f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45700/0 (socket says 172.21.15.50:45700) 2023-10-19T10:51:33.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 -- 172.21.15.50:0/3614467131 learned_addr learned my addr 172.21.15.50:0/3614467131 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:51:33.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff2537fe640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540d0e70 0x7ff2540c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff258cbc640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff25415c680 0x7ff2541f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 -- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540d0e70 msgr2=0x7ff2540c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540d0e70 0x7ff2540c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 -- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff25415c680 msgr2=0x7ff2541f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:33.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff25415c680 0x7ff2541f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:33.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff253fff640 1 -- 172.21.15.50:0/3614467131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009b30 con 0x7ff2540cd960 2023-10-19T10:51:33.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff258cbc640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff25415c680 0x7ff2541f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:51:33.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.807+0000 7ff2537fe640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2540d0e70 0x7ff2540c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:51:33.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff253fff640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2540cd960 0x7ff2541f48e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e870 tx=0x7ff24400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:51:33.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff250ff9640 1 -- 172.21.15.50:0/3614467131 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440095c0 con 0x7ff2540cd960 2023-10-19T10:51:33.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff250ff9640 1 -- 172.21.15.50:0/3614467131 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24400c940 con 0x7ff2540cd960 2023-10-19T10:51:33.820 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff250ff9640 1 -- 172.21.15.50:0/3614467131 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24400ccb0 con 0x7ff2540cd960 2023-10-19T10:51:33.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff25a71f640 1 -- 172.21.15.50:0/3614467131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff254156870 con 0x7ff2540cd960 2023-10-19T10:51:33.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff25a71f640 1 -- 172.21.15.50:0/3614467131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff254156e30 con 0x7ff2540cd960 2023-10-19T10:51:33.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.808+0000 7ff25a71f640 1 -- 172.21.15.50:0/3614467131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff254156fc0 con 0x7ff2540cd960 2023-10-19T10:51:33.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.810+0000 7ff250ff9640 1 -- 172.21.15.50:0/3614467131 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff244013470 con 0x7ff2540cd960 2023-10-19T10:51:33.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.810+0000 7ff250ff9640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff23406a080 0x7ff23406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:51:33.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.810+0000 7ff250ff9640 1 -- 172.21.15.50:0/3614467131 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff244089230 con 0x7ff2540cd960 2023-10-19T10:51:33.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.810+0000 7ff250ff9640 1 -- 172.21.15.50:0/3614467131 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7ff2440896a0 con 0x7ff2540cd960 2023-10-19T10:51:33.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.810+0000 7ff2537fe640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff23406a080 0x7ff23406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:51:33.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.811+0000 7ff25a71f640 1 -- 172.21.15.50:0/3614467131 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff254066e80 con 0x7ff2540cd960 2023-10-19T10:51:33.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:33.811+0000 7ff2537fe640 1 --2- 172.21.15.50:0/3614467131 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff23406a080 0x7ff23406c540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff254153270 tx=0x7ff23c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:51:34.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.004+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2500d1530 con 0x7fc2500d2910 2023-10-19T10:51:34.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.004+0000 7fc245ffb640 1 -- 172.21.15.50:0/4052571595 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fc240057060 con 0x7fc2500d2910 2023-10-19T10:51:34.008 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:51:34.008 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:51:34.009 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:51:34.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.006+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc23406a050 msgr2=0x7fc23406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:34.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.006+0000 7fc257a0b640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc23406a050 0x7fc23406c510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc24c035000 tx=0x7fc24c008030 comp rx=0 tx=0).stop 2023-10-19T10:51:34.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.006+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 msgr2=0x7fc2501e9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:51:34.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.006+0000 7fc257a0b640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2501e9eb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c430 tx=0x7fc24000c8f0 comp rx=0 tx=0).stop 2023-10-19T10:51:34.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 shutdown_connections 2023-10-19T10:51:34.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc23406a050 0x7fc23406c510 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:34.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2500d2cf0 0x7fc2501ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:34.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2500d2910 0x7fc2501e9eb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:34.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 --2- 172.21.15.50:0/4052571595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2500d0fc0 0x7fc250155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:51:34.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 >> 172.21.15.50:0/4052571595 conn(0x7fc250089ff0 msgr2=0x7fc250163ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:51:34.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 shutdown_connections 2023-10-19T10:51:34.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:51:34.007+0000 7fc257a0b640 1 -- 172.21.15.50:0/4052571595 wait complete. 2023-10-19T10:51:34.020 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-19T10:51:34.142 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":5186,"entity":{"name":{"type":"client","num":5186},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":757114590}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":773.67724452200002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.16736686642890616,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5186 192.168.0.1:0/757114590","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"6f3e6b48891b9b50415c0822b6b870797fcda16d","ceph_version":"ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)","entity_id":"1","hostname":"smithi183","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"95663","root":"/"}}] 2023-10-19T10:51:34.143 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi183", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "95663", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 757114590, "type": "any" }, "name": { "num": 5186, "type": "client" } }, "id": 5186, "inst": "client.5186 192.168.0.1:0/757114590", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.16736686642890616 }, "state": "open", "uptime": 773.677244522 } ] 2023-10-19T10:51:34.244 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-19T10:51:58.737 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:51:58.741 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:52:28.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:52:28.806 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:52:34.258 ERROR:tasks.ceph_test_case:Log output: 2023-10-19T10:52:34.259 WARNING:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-19T10:52:34.259 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:52:34.260 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:52:34.260 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:34.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:34.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.473+0000 7f2baf9f7640 1 Processor -- start 2023-10-19T10:52:34.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.473+0000 7f2baf9f7640 1 -- start start 2023-10-19T10:52:34.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2baf9f7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2ba815a370 0x7f2ba806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2baf9f7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2ba80b23c0 0x7f2ba815e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2baf9f7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba80caf80 con 0x7f2ba80b1f30 2023-10-19T10:52:34.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2baf9f7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba80d02e0 con 0x7f2ba815a370 2023-10-19T10:52:34.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2baf9f7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba80b2850 con 0x7f2ba80b23c0 2023-10-19T10:52:34.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2bad793640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2ba815a370 0x7f2ba806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2bacf92640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2ba80b1f30 0x7f2ba815de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:51486/0 (socket says 172.21.15.50:51486) 2023-10-19T10:52:34.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.474+0000 7f2bacf92640 1 -- 172.21.15.50:0/193140915 learned_addr learned my addr 172.21.15.50:0/193140915 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:34.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2badf94640 1 --2- 172.21.15.50:0/193140915 >> v2:172.21.15.183:3300/0 conn(0x7f2ba80b23c0 0x7f2ba815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2bad793640 1 -- 172.21.15.50:0/193140915 >> v2:172.21.15.183:3300/0 conn(0x7f2ba80b23c0 msgr2=0x7f2ba815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:34.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2bad793640 1 --2- 172.21.15.50:0/193140915 >> v2:172.21.15.183:3300/0 conn(0x7f2ba80b23c0 0x7f2ba815e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2bad793640 1 -- 172.21.15.50:0/193140915 >> v1:172.21.15.50:6789/0 conn(0x7f2ba80b1f30 legacy=0x7f2ba815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2b9ffff640 1 -- 172.21.15.50:0/193140915 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483259458 0 0) 0x7f2ba80caf80 con 0x7f2ba80b1f30 2023-10-19T10:52:34.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2bad793640 1 -- 172.21.15.50:0/193140915 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba815a750 con 0x7f2ba815a370 2023-10-19T10:52:34.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.475+0000 7f2bad793640 1 --2- 172.21.15.50:0/193140915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba815a370 0x7f2ba806fc80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80cce10 tx=0x7f2ba402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff6a86dd57e526b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:34.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2b9ffff640 1 -- 172.21.15.50:0/193140915 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4031830 con 0x7f2ba815a370 2023-10-19T10:52:34.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2b9ffff640 1 -- 172.21.15.50:0/193140915 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4031990 con 0x7f2ba815a370 2023-10-19T10:52:34.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2b9ffff640 1 -- 172.21.15.50:0/193140915 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4031d20 con 0x7f2ba815a370 2023-10-19T10:52:34.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 -- 172.21.15.50:0/193140915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba815a370 msgr2=0x7f2ba806fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/193140915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba815a370 0x7f2ba806fc80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80cce10 tx=0x7f2ba402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 -- 172.21.15.50:0/193140915 shutdown_connections 2023-10-19T10:52:34.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/193140915 >> v2:172.21.15.183:3300/0 conn(0x7f2ba80b23c0 0x7f2ba815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/193140915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba815a370 0x7f2ba806fc80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 -- 172.21.15.50:0/193140915 >> 172.21.15.50:0/193140915 conn(0x7f2ba80ccf30 msgr2=0x7f2ba80cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:34.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 -- 172.21.15.50:0/193140915 shutdown_connections 2023-10-19T10:52:34.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.476+0000 7f2baf9f7640 1 -- 172.21.15.50:0/193140915 wait complete. 2023-10-19T10:52:34.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.477+0000 7f2baf9f7640 1 Processor -- start 2023-10-19T10:52:34.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.477+0000 7f2baf9f7640 1 -- start start 2023-10-19T10:52:34.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2baf9f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 0x7f2ba814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2baf9f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba80b23c0 0x7f2ba80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2baf9f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ba815a370 0x7f2ba80d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2baf9f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba81f05f0 con 0x7f2ba80b1f30 2023-10-19T10:52:34.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2baf9f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba81f2d70 con 0x7f2ba80b23c0 2023-10-19T10:52:34.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2baf9f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba81f3050 con 0x7f2ba815a370 2023-10-19T10:52:34.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2bad793640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 0x7f2ba814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2bad793640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 0x7f2ba814c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48808/0 (socket says 172.21.15.50:48808) 2023-10-19T10:52:34.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.478+0000 7f2bad793640 1 -- 172.21.15.50:0/1662123558 learned_addr learned my addr 172.21.15.50:0/1662123558 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:34.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bad793640 1 -- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ba815a370 msgr2=0x7f2ba80d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bad793640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ba815a370 0x7f2ba80d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bad793640 1 -- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba80b23c0 msgr2=0x7f2ba80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bacf92640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba80b23c0 0x7f2ba80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bad793640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba80b23c0 0x7f2ba80d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bad793640 1 -- 172.21.15.50:0/1662123558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4009400 con 0x7f2ba80b1f30 2023-10-19T10:52:34.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2bad793640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 0x7f2ba814c200 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009890 tx=0x7f2ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:34.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba403ba70 con 0x7f2ba80b1f30 2023-10-19T10:52:34.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba403bbf0 con 0x7f2ba80b1f30 2023-10-19T10:52:34.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba81f3330 con 0x7f2ba80b1f30 2023-10-19T10:52:34.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4045400 con 0x7f2ba80b1f30 2023-10-19T10:52:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.479+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba814c740 con 0x7f2ba80b1f30 2023-10-19T10:52:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.480+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba81f3850 con 0x7f2ba80b1f30 2023-10-19T10:52:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.481+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2ba404e440 con 0x7f2ba80b1f30 2023-10-19T10:52:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.481+0000 7f2b9dffb640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2b8c06a050 0x7f2b8c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.481+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f2ba40aaa60 con 0x7f2ba80b1f30 2023-10-19T10:52:34.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.481+0000 7f2bacf92640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2b8c06a050 0x7f2b8c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.482+0000 7f2bacf92640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2b8c06a050 0x7f2b8c06c510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2ba81f08f0 tx=0x7f2b98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:34.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.486+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2ba81f3850 con 0x7f2ba80b1f30 2023-10-19T10:52:34.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.663+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ba81f3850 con 0x7f2ba80b1f30 2023-10-19T10:52:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.664+0000 7f2b9dffb640 1 -- 172.21.15.50:0/1662123558 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18701 (secure 0 0 0) 0x7f2ba81f3850 con 0x7f2ba80b1f30 2023-10-19T10:52:34.668 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:34.669 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:46:26.319171+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2b8c06a050 msgr2=0x7f2b8c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2b8c06a050 0x7f2b8c06c510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2ba81f08f0 tx=0x7f2b98009040 comp rx=0 tx=0).stop 2023-10-19T10:52:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 msgr2=0x7f2ba814c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 0x7f2ba814c200 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009890 tx=0x7f2ba40046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 shutdown_connections 2023-10-19T10:52:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2b8c06a050 0x7f2b8c06c510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ba815a370 0x7f2ba80d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba80b23c0 0x7f2ba80d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 --2- 172.21.15.50:0/1662123558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ba80b1f30 0x7f2ba814c200 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 >> 172.21.15.50:0/1662123558 conn(0x7f2ba80ccf30 msgr2=0x7f2ba80b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:34.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 shutdown_connections 2023-10-19T10:52:34.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.666+0000 7f2baf9f7640 1 -- 172.21.15.50:0/1662123558 wait complete. 2023-10-19T10:52:34.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:38:26.398890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:38:26.910608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:52:34.682 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:52:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.787+0000 7fa88b80b640 1 Processor -- start 2023-10-19T10:52:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.787+0000 7fa88b80b640 1 -- start start 2023-10-19T10:52:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa88b80b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa88415c6f0 0x7fa8840ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa88b80b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa8840b1f60 0x7fa8840ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa88b80b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8840afab0 0x7fa884160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa88b80b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88406f2a0 con 0x7fa88415c6f0 2023-10-19T10:52:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa88b80b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88406fe20 con 0x7fa8840b1f60 2023-10-19T10:52:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa88b80b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840702b0 con 0x7fa8840afab0 2023-10-19T10:52:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa8895a7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa88415c6f0 0x7fa8840ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa8895a7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa88415c6f0 0x7fa8840ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48820/0 (socket says 172.21.15.50:48820) 2023-10-19T10:52:34.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa8895a7640 1 -- 172.21.15.50:0/4293023881 learned_addr learned my addr 172.21.15.50:0/4293023881 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:34.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.788+0000 7fa889da8640 1 --2- 172.21.15.50:0/4293023881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8840afab0 0x7fa884160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa888da6640 1 --2- 172.21.15.50:0/4293023881 >> v2:172.21.15.183:3300/0 conn(0x7fa8840b1f60 0x7fa8840ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa8895a7640 1 -- 172.21.15.50:0/4293023881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8840afab0 msgr2=0x7fa884160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa8895a7640 1 --2- 172.21.15.50:0/4293023881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8840afab0 0x7fa884160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa8895a7640 1 -- 172.21.15.50:0/4293023881 >> v2:172.21.15.183:3300/0 conn(0x7fa8840b1f60 msgr2=0x7fa8840ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa8895a7640 1 --2- 172.21.15.50:0/4293023881 >> v2:172.21.15.183:3300/0 conn(0x7fa8840b1f60 0x7fa8840ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa8895a7640 1 -- 172.21.15.50:0/4293023881 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88406f4f0 con 0x7fa88415c6f0 2023-10-19T10:52:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa8895a7640 1 --2- 172.21.15.50:0/4293023881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88415c6f0 0x7fa8840ce020 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa8840c3b60 tx=0x7fa87c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cee7336220e8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa883fff640 1 -- 172.21.15.50:0/4293023881 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c031870 con 0x7fa88415c6f0 2023-10-19T10:52:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa883fff640 1 -- 172.21.15.50:0/4293023881 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0319d0 con 0x7fa88415c6f0 2023-10-19T10:52:34.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.789+0000 7fa883fff640 1 -- 172.21.15.50:0/4293023881 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c031d60 con 0x7fa88415c6f0 2023-10-19T10:52:34.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 -- 172.21.15.50:0/4293023881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88415c6f0 msgr2=0x7fa8840ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 --2- 172.21.15.50:0/4293023881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88415c6f0 0x7fa8840ce020 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa8840c3b60 tx=0x7fa87c02b910 comp rx=0 tx=0).stop 2023-10-19T10:52:34.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 -- 172.21.15.50:0/4293023881 shutdown_connections 2023-10-19T10:52:34.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 --2- 172.21.15.50:0/4293023881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8840afab0 0x7fa884160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 --2- 172.21.15.50:0/4293023881 >> v2:172.21.15.183:3300/0 conn(0x7fa8840b1f60 0x7fa8840ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 --2- 172.21.15.50:0/4293023881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88415c6f0 0x7fa8840ce020 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 -- 172.21.15.50:0/4293023881 >> 172.21.15.50:0/4293023881 conn(0x7fa8840ccf60 msgr2=0x7fa8840cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:34.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 -- 172.21.15.50:0/4293023881 shutdown_connections 2023-10-19T10:52:34.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.790+0000 7fa88b80b640 1 -- 172.21.15.50:0/4293023881 wait complete. 2023-10-19T10:52:34.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.791+0000 7fa88b80b640 1 Processor -- start 2023-10-19T10:52:34.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.791+0000 7fa88b80b640 1 -- start start 2023-10-19T10:52:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa88b80b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 0x7fa8841f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa88b80b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa88415c6f0 0x7fa88415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa88b80b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa884154bc0 0x7fa8841f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa88b80b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88415e3b0 con 0x7fa8840afab0 2023-10-19T10:52:34.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa88b80b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88415e0d0 con 0x7fa88415c6f0 2023-10-19T10:52:34.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa88b80b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa884159470 con 0x7fa884154bc0 2023-10-19T10:52:34.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 0x7fa8841f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 0x7fa8841f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48824/0 (socket says 172.21.15.50:48824) 2023-10-19T10:52:34.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 -- 172.21.15.50:0/3231773554 learned_addr learned my addr 172.21.15.50:0/3231773554 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:34.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa888da6640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa88415c6f0 0x7fa88415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa889da8640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa884154bc0 0x7fa8841f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 -- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa884154bc0 msgr2=0x7fa8841f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa884154bc0 0x7fa8841f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 -- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa88415c6f0 msgr2=0x7fa88415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa88415c6f0 0x7fa88415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.792+0000 7fa8895a7640 1 -- 172.21.15.50:0/3231773554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0094d0 con 0x7fa8840afab0 2023-10-19T10:52:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.793+0000 7fa8895a7640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 0x7fa8841f1c60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa87c02b950 tx=0x7fa87c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.793+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c031860 con 0x7fa8840afab0 2023-10-19T10:52:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.793+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0443f0 con 0x7fa8840afab0 2023-10-19T10:52:34.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.793+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa884159750 con 0x7fa8840afab0 2023-10-19T10:52:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.793+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0447a0 con 0x7fa8840afab0 2023-10-19T10:52:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.793+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa884154700 con 0x7fa8840afab0 2023-10-19T10:52:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.794+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8841592d0 con 0x7fa8840afab0 2023-10-19T10:52:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.794+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa87c04d450 con 0x7fa8840afab0 2023-10-19T10:52:34.807 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.795+0000 7fa881ffb640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa87006a050 0x7fa87006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.795+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa87c02d040 con 0x7fa8840afab0 2023-10-19T10:52:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.795+0000 7fa888da6640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa87006a050 0x7fa87006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.796+0000 7fa888da6640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa87006a050 0x7fa87006c510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa878006fd0 tx=0x7fa878008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.799+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa8841592d0 con 0x7fa8840afab0 2023-10-19T10:52:34.979 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.976+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa884159c70 con 0x7fa8840afab0 2023-10-19T10:52:34.980 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.976+0000 7fa881ffb640 1 -- 172.21.15.50:0/3231773554 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa884159c70 con 0x7fa8840afab0 2023-10-19T10:52:34.981 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:52:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa87006a050 msgr2=0x7fa87006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa87006a050 0x7fa87006c510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa878006fd0 tx=0x7fa878008040 comp rx=0 tx=0).stop 2023-10-19T10:52:34.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 msgr2=0x7fa8841f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:34.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 0x7fa8841f1c60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa87c02b950 tx=0x7fa87c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 shutdown_connections 2023-10-19T10:52:34.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa87006a050 0x7fa87006c510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.983 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa884154bc0 0x7fa8841f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa88415c6f0 0x7fa88415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 --2- 172.21.15.50:0/3231773554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8840afab0 0x7fa8841f1c60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:34.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 >> 172.21.15.50:0/3231773554 conn(0x7fa8840ccf60 msgr2=0x7fa8840b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:34.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 shutdown_connections 2023-10-19T10:52:34.984 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:34.978+0000 7fa88b80b640 1 -- 172.21.15.50:0/3231773554 wait complete. 2023-10-19T10:52:34.993 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:52:35.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.098+0000 7f200077f640 1 Processor -- start 2023-10-19T10:52:35.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.098+0000 7f200077f640 1 -- start start 2023-10-19T10:52:35.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f200077f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1ff8069800 0x7f1ff80cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f200077f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815cc80 0x7f1ff806dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f200077f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff8160580 con 0x7f1ff806ef00 2023-10-19T10:52:35.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f200077f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff8160860 con 0x7f1ff8069800 2023-10-19T10:52:35.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f200077f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8160b40 con 0x7f1ff815cc80 2023-10-19T10:52:35.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffe51b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1ff8069800 0x7f1ff80cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffdd1a640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1ff806ef00 0x7f1ff8069be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:51498/0 (socket says 172.21.15.50:51498) 2023-10-19T10:52:35.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffe51b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1ff8069800 0x7f1ff80cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48830/0 (socket says 172.21.15.50:48830) 2023-10-19T10:52:35.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffdd1a640 1 -- 172.21.15.50:0/3740527210 learned_addr learned my addr 172.21.15.50:0/3740527210 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:35.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffed1c640 1 --2- 172.21.15.50:0/3740527210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815cc80 0x7f1ff806dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffe51b640 1 -- 172.21.15.50:0/3740527210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815cc80 msgr2=0x7f1ff806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:35.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffe51b640 1 --2- 172.21.15.50:0/3740527210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815cc80 0x7f1ff806dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffe51b640 1 -- 172.21.15.50:0/3740527210 >> v1:172.21.15.50:6789/0 conn(0x7f1ff806ef00 legacy=0x7f1ff8069be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffd519640 1 -- 172.21.15.50:0/3740527210 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708295680 0 0) 0x7f1ff8160580 con 0x7f1ff806ef00 2023-10-19T10:52:35.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.099+0000 7f1ffe51b640 1 -- 172.21.15.50:0/3740527210 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8160e20 con 0x7f1ff8069800 2023-10-19T10:52:35.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.100+0000 7f1ffe51b640 1 --2- 172.21.15.50:0/3740527210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8069800 0x7f1ff80cfc60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40097a0 tx=0x7f1ff402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd10467b3adf54e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.100+0000 7f1ffd519640 1 -- 172.21.15.50:0/3740527210 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4031830 con 0x7f1ff8069800 2023-10-19T10:52:35.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.100+0000 7f1ffd519640 1 -- 172.21.15.50:0/3740527210 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff4031990 con 0x7f1ff8069800 2023-10-19T10:52:35.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.100+0000 7f1ffd519640 1 -- 172.21.15.50:0/3740527210 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4031d20 con 0x7f1ff8069800 2023-10-19T10:52:35.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.100+0000 7f200077f640 1 -- 172.21.15.50:0/3740527210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8069800 msgr2=0x7f1ff80cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.100+0000 7f200077f640 1 --2- 172.21.15.50:0/3740527210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8069800 0x7f1ff80cfc60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40097a0 tx=0x7f1ff402b8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.101+0000 7f200077f640 1 -- 172.21.15.50:0/3740527210 shutdown_connections 2023-10-19T10:52:35.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.101+0000 7f200077f640 1 --2- 172.21.15.50:0/3740527210 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff815cc80 0x7f1ff806dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.101+0000 7f200077f640 1 --2- 172.21.15.50:0/3740527210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8069800 0x7f1ff80cfc60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.101+0000 7f200077f640 1 -- 172.21.15.50:0/3740527210 >> 172.21.15.50:0/3740527210 conn(0x7f1ff80cf170 msgr2=0x7f1ff80cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.101+0000 7f200077f640 1 -- 172.21.15.50:0/3740527210 shutdown_connections 2023-10-19T10:52:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.101+0000 7f200077f640 1 -- 172.21.15.50:0/3740527210 wait complete. 2023-10-19T10:52:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.102+0000 7f200077f640 1 Processor -- start 2023-10-19T10:52:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.102+0000 7f200077f640 1 -- start start 2023-10-19T10:52:35.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.102+0000 7f200077f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff8069800 0x7f1ff80c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.102+0000 7f200077f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 0x7f1ff815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f200077f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815cc80 0x7f1ff815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f200077f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80c8ee0 con 0x7f1ff806ef00 2023-10-19T10:52:35.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f200077f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80c92e0 con 0x7f1ff815cc80 2023-10-19T10:52:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f200077f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80c8540 con 0x7f1ff8069800 2023-10-19T10:52:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 0x7f1ff815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 0x7f1ff815e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48836/0 (socket says 172.21.15.50:48836) 2023-10-19T10:52:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 -- 172.21.15.50:0/863506308 learned_addr learned my addr 172.21.15.50:0/863506308 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:35.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffed1c640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815cc80 0x7f1ff815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffe51b640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff8069800 0x7f1ff80c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 -- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff8069800 msgr2=0x7f1ff80c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff8069800 0x7f1ff80c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 -- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815cc80 msgr2=0x7f1ff815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815cc80 0x7f1ff815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 -- 172.21.15.50:0/863506308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4009400 con 0x7f1ff806ef00 2023-10-19T10:52:35.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffed1c640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815cc80 0x7f1ff815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:35.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1ffdd1a640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 0x7f1ff815e050 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800c4d0 tx=0x7f1fe800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80074e0 con 0x7f1ff806ef00 2023-10-19T10:52:35.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8007660 con 0x7f1ff806ef00 2023-10-19T10:52:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff80c8820 con 0x7f1ff806ef00 2023-10-19T10:52:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004a20 con 0x7f1ff806ef00 2023-10-19T10:52:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.103+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff80c81f0 con 0x7f1ff806ef00 2023-10-19T10:52:35.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.104+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff81422c0 con 0x7f1ff806ef00 2023-10-19T10:52:35.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.105+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1fe8004b80 con 0x7f1ff806ef00 2023-10-19T10:52:35.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.105+0000 7f1feeffd640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fdc06a050 0x7f1fdc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.105+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1fe8087860 con 0x7f1ff806ef00 2023-10-19T10:52:35.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.106+0000 7f1ffe51b640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fdc06a050 0x7f1fdc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.106+0000 7f1ffe51b640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fdc06a050 0x7f1fdc06c510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4036000 tx=0x7f1ff4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.110+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1fe8052e30 con 0x7f1ff806ef00 2023-10-19T10:52:35.287 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.283+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ff804e420 con 0x7f1ff806ef00 2023-10-19T10:52:35.288 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.284+0000 7f1feeffd640 1 -- 172.21.15.50:0/863506308 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1fe8018020 con 0x7f1ff806ef00 2023-10-19T10:52:35.288 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:52:35.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fdc06a050 msgr2=0x7f1fdc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.290 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fdc06a050 0x7f1fdc06c510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4036000 tx=0x7f1ff4008030 comp rx=0 tx=0).stop 2023-10-19T10:52:35.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 msgr2=0x7f1ff815e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 0x7f1ff815e050 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800c4d0 tx=0x7f1fe800c990 comp rx=0 tx=0).stop 2023-10-19T10:52:35.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 shutdown_connections 2023-10-19T10:52:35.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fdc06a050 0x7f1fdc06c510 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.291 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff815cc80 0x7f1ff815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff806ef00 0x7f1ff815e050 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 --2- 172.21.15.50:0/863506308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ff8069800 0x7f1ff80c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 >> 172.21.15.50:0/863506308 conn(0x7f1ff80cf170 msgr2=0x7f1ff80cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:35.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 shutdown_connections 2023-10-19T10:52:35.292 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.286+0000 7f200077f640 1 -- 172.21.15.50:0/863506308 wait complete. 2023-10-19T10:52:35.301 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:52:35.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.407+0000 7f971d770640 1 Processor -- start 2023-10-19T10:52:35.411 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.407+0000 7f971d770640 1 -- start start 2023-10-19T10:52:35.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f971d770640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f97180d2ac0 0x7f97180cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f971d770640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97180b7ca0 con 0x7f97180b10b0 2023-10-19T10:52:35.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f971d770640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97180b7ef0 con 0x7f971815c510 2023-10-19T10:52:35.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f971d770640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97180d21c0 con 0x7f97180d2ac0 2023-10-19T10:52:35.412 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f97167fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f97180d2ac0 0x7f97180cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f97177fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f97180b10b0 0x7f97180d06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:51514/0 (socket says 172.21.15.50:51514) 2023-10-19T10:52:35.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f97167fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f97180d2ac0 0x7f97180cfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48846/0 (socket says 172.21.15.50:48846) 2023-10-19T10:52:35.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f97177fe640 1 -- 172.21.15.50:0/1477958860 learned_addr learned my addr 172.21.15.50:0/1477958860 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:35.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.408+0000 7f97167fc640 1 -- 172.21.15.50:0/1477958860 >> v1:172.21.15.183:6789/0 conn(0x7f971815c510 legacy=0x7f97180d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:35.413 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f97167fc640 1 -- 172.21.15.50:0/1477958860 >> v1:172.21.15.50:6789/0 conn(0x7f97180b10b0 legacy=0x7f97180d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f97167fc640 1 -- 172.21.15.50:0/1477958860 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180b81d0 con 0x7f97180d2ac0 2023-10-19T10:52:35.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f9715ffb640 1 -- 172.21.15.50:0/1477958860 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197469126 0 0) 0x7f97180b7ca0 con 0x7f97180b10b0 2023-10-19T10:52:35.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f97167fc640 1 --2- 172.21.15.50:0/1477958860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97180d2ac0 0x7f97180cfff0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f97000097d0 tx=0x7f970002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ed2cdd74850b24 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f9715ffb640 1 -- 172.21.15.50:0/1477958860 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700031830 con 0x7f97180d2ac0 2023-10-19T10:52:35.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f9715ffb640 1 -- 172.21.15.50:0/1477958860 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9700031990 con 0x7f97180d2ac0 2023-10-19T10:52:35.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.409+0000 7f9715ffb640 1 -- 172.21.15.50:0/1477958860 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700031d20 con 0x7f97180d2ac0 2023-10-19T10:52:35.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 -- 172.21.15.50:0/1477958860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97180d2ac0 msgr2=0x7f97180cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 --2- 172.21.15.50:0/1477958860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97180d2ac0 0x7f97180cfff0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f97000097d0 tx=0x7f970002b8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 -- 172.21.15.50:0/1477958860 shutdown_connections 2023-10-19T10:52:35.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 --2- 172.21.15.50:0/1477958860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97180d2ac0 0x7f97180cfff0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 -- 172.21.15.50:0/1477958860 >> 172.21.15.50:0/1477958860 conn(0x7f9718071320 msgr2=0x7f9718071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:35.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 -- 172.21.15.50:0/1477958860 shutdown_connections 2023-10-19T10:52:35.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.410+0000 7f971d770640 1 -- 172.21.15.50:0/1477958860 wait complete. 2023-10-19T10:52:35.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.411+0000 7f971d770640 1 Processor -- start 2023-10-19T10:52:35.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.411+0000 7f971d770640 1 -- start start 2023-10-19T10:52:35.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f971d770640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97180b10b0 0x7f97180c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f971d770640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97180d2ac0 0x7f97181563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f971d770640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 0x7f9718156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f971d770640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181560f0 con 0x7f971815c510 2023-10-19T10:52:35.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f971d770640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180ca560 con 0x7f97180b10b0 2023-10-19T10:52:35.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f971d770640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97180c89c0 con 0x7f97180d2ac0 2023-10-19T10:52:35.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f97177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 0x7f9718156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f97177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 0x7f9718156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48852/0 (socket says 172.21.15.50:48852) 2023-10-19T10:52:35.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f97177fe640 1 -- 172.21.15.50:0/2268030141 learned_addr learned my addr 172.21.15.50:0/2268030141 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:35.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f97167fc640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97180d2ac0 0x7f97181563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.412+0000 7f9716ffd640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97180b10b0 0x7f97180c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f97177fe640 1 -- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97180d2ac0 msgr2=0x7f97181563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f97177fe640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97180d2ac0 0x7f97181563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f97177fe640 1 -- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97180b10b0 msgr2=0x7f97180c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f97177fe640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97180b10b0 0x7f97180c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f97177fe640 1 -- 172.21.15.50:0/2268030141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700009400 con 0x7f971815c510 2023-10-19T10:52:35.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f97177fe640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 0x7f9718156930 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f971000c450 tx=0x7f971000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97100074e0 con 0x7f971815c510 2023-10-19T10:52:35.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9710007660 con 0x7f971815c510 2023-10-19T10:52:35.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971815efc0 con 0x7f971815c510 2023-10-19T10:52:35.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710004a20 con 0x7f971815c510 2023-10-19T10:52:35.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.413+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97180c9f10 con 0x7f971815c510 2023-10-19T10:52:35.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.414+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97180ca6d0 con 0x7f971815c510 2023-10-19T10:52:35.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.415+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9710004b80 con 0x7f971815c510 2023-10-19T10:52:35.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.415+0000 7f970f7fe640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f96f806a050 0x7f96f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.415+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f9710087c10 con 0x7f971815c510 2023-10-19T10:52:35.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.415+0000 7f9716ffd640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f96f806a050 0x7f96f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.416+0000 7f9716ffd640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f96f806a050 0x7f96f806c510 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fd0 tx=0x7f970800d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.420+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9710053230 con 0x7f971815c510 2023-10-19T10:52:35.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.592+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f971804e420 con 0x7f971815c510 2023-10-19T10:52:35.596 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.593+0000 7f970f7fe640 1 -- 172.21.15.50:0/2268030141 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9710018020 con 0x7f971815c510 2023-10-19T10:52:35.597 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:52:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f96f806a050 msgr2=0x7f96f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f96f806a050 0x7f96f806c510 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fd0 tx=0x7f970800d040 comp rx=0 tx=0).stop 2023-10-19T10:52:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 msgr2=0x7f9718156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 0x7f9718156930 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f971000c450 tx=0x7f971000c910 comp rx=0 tx=0).stop 2023-10-19T10:52:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 shutdown_connections 2023-10-19T10:52:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f96f806a050 0x7f96f806c510 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f971815c510 0x7f9718156930 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97180d2ac0 0x7f97181563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 --2- 172.21.15.50:0/2268030141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97180b10b0 0x7f97180c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 >> 172.21.15.50:0/2268030141 conn(0x7f9718071320 msgr2=0x7f9718071700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 shutdown_connections 2023-10-19T10:52:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.595+0000 7f971d770640 1 -- 172.21.15.50:0/2268030141 wait complete. 2023-10-19T10:52:35.609 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:52:35.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.716+0000 7fbe2b1f7640 1 Processor -- start 2023-10-19T10:52:35.720 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.716+0000 7fbe2b1f7640 1 -- start start 2023-10-19T10:52:35.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe2b1f7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbe240b38f0 0x7fbe240d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.723 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe2b1f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe24161210 0x7fbe240b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe2b1f7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe240af250 con 0x7fbe240b3090 2023-10-19T10:52:35.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe2b1f7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe240b0190 con 0x7fbe240b38f0 2023-10-19T10:52:35.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe2b1f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240af510 con 0x7fbe24161210 2023-10-19T10:52:35.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe23fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fbe240b3090 0x7fbe24160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:51520/0 (socket says 172.21.15.50:51520) 2023-10-19T10:52:35.724 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.717+0000 7fbe23fff640 1 -- 172.21.15.50:0/55978987 learned_addr learned my addr 172.21.15.50:0/55978987 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:35.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe28f93640 1 --2- 172.21.15.50:0/55978987 >> v2:172.21.15.183:3300/0 conn(0x7fbe240b38f0 0x7fbe240d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe29794640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe24161210 0x7fbe240b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783339115 0 0) 0x7fbe240af250 con 0x7fbe240b3090 2023-10-19T10:52:35.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe0c003520 con 0x7fbe240b3090 2023-10-19T10:52:35.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 720499721 0 0) 0x7fbe0c003520 con 0x7fbe240b3090 2023-10-19T10:52:35.725 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe24161210 msgr2=0x7fbe240b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:35.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe24161210 0x7fbe240b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 >> v2:172.21.15.183:3300/0 conn(0x7fbe240b38f0 msgr2=0x7fbe240d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:35.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 --2- 172.21.15.50:0/55978987 >> v2:172.21.15.183:3300/0 conn(0x7fbe240b38f0 0x7fbe240d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.726 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240af7f0 con 0x7fbe240b3090 2023-10-19T10:52:35.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fbe18001670 con 0x7fbe240b3090 2023-10-19T10:52:35.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe28f93640 1 --2- 172.21.15.50:0/55978987 >> v2:172.21.15.183:3300/0 conn(0x7fbe240b38f0 0x7fbe240d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:35.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 >> v1:172.21.15.50:6789/0 conn(0x7fbe240b3090 legacy=0x7fbe24160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.727 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0c005180 0x7fbe0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240b0190 con 0x7fbe0c003970 2023-10-19T10:52:35.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240af510 con 0x7fbe0c005180 2023-10-19T10:52:35.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c00bec0 con 0x7fbe0c0047c0 2023-10-19T10:52:35.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe29794640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe24161210 0x7fbe240b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:35.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe28f93640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe23fff640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe28f93640 1 -- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c0047c0 msgr2=0x7fbe0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:35.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe28f93640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c0047c0 0x7fbe0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe28f93640 1 -- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0c005180 msgr2=0x7fbe0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:35.729 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.718+0000 7fbe29794640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0c005180 0x7fbe0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe28f93640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0c005180 0x7fbe0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe28f93640 1 -- 172.21.15.50:0/55978987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c003520 con 0x7fbe0c003970 2023-10-19T10:52:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe29794640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0c005180 0x7fbe0c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:35.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe28f93640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00e350 tx=0x7fbe0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c8177785dba542 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c02ce30 con 0x7fbe0c003970 2023-10-19T10:52:35.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c00f040 con 0x7fbe0c003970 2023-10-19T10:52:35.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.719+0000 7fbe237fe640 1 -- 172.21.15.50:0/55978987 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c034640 con 0x7fbe0c003970 2023-10-19T10:52:35.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/55978987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe0c003970 msgr2=0x7fbe0c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/55978987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe0c003970 0x7fbe0c003d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00e350 tx=0x7fbe0c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe29794640 1 -- 172.21.15.50:0/55978987 reap_dead start 2023-10-19T10:52:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/55978987 shutdown_connections 2023-10-19T10:52:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/55978987 >> 172.21.15.50:0/55978987 conn(0x7fbe2409b960 msgr2=0x7fbe240d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/55978987 shutdown_connections 2023-10-19T10:52:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.720+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/55978987 wait complete. 2023-10-19T10:52:35.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.721+0000 7fbe2b1f7640 1 Processor -- start 2023-10-19T10:52:35.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.721+0000 7fbe2b1f7640 1 -- start start 2023-10-19T10:52:35.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe2b1f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c003970 0x7fbe2415b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe2b1f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 0x7fbe2406c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe2b1f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe2415cf50 0x7fbe2415d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe2b1f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240cc8c0 con 0x7fbe2406bc80 2023-10-19T10:52:35.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe2b1f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2406c620 con 0x7fbe2415cf50 2023-10-19T10:52:35.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe2b1f7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240ca770 con 0x7fbe0c003970 2023-10-19T10:52:35.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe23fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 0x7fbe2406c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe23fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 0x7fbe2406c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48868/0 (socket says 172.21.15.50:48868) 2023-10-19T10:52:35.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.722+0000 7fbe23fff640 1 -- 172.21.15.50:0/3619406290 learned_addr learned my addr 172.21.15.50:0/3619406290 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:35.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe29794640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe2415cf50 0x7fbe2415d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe23fff640 1 -- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c003970 msgr2=0x7fbe2415b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe28f93640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c003970 0x7fbe2415b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe23fff640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c003970 0x7fbe2415b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe23fff640 1 -- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe2415cf50 msgr2=0x7fbe2415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe23fff640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe2415cf50 0x7fbe2415d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe23fff640 1 -- 172.21.15.50:0/3619406290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240cdee0 con 0x7fbe2406bc80 2023-10-19T10:52:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe23fff640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 0x7fbe2406c0e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e850 tx=0x7fbe1800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800c870 con 0x7fbe2406bc80 2023-10-19T10:52:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1800c9d0 con 0x7fbe2406bc80 2023-10-19T10:52:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe240ca1b0 con 0x7fbe2406bc80 2023-10-19T10:52:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800cd80 con 0x7fbe2406bc80 2023-10-19T10:52:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.723+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe240cdb70 con 0x7fbe2406bc80 2023-10-19T10:52:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.724+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe240cdd30 con 0x7fbe2406bc80 2023-10-19T10:52:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.724+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbe1801d460 con 0x7fbe2406bc80 2023-10-19T10:52:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.725+0000 7fbe217fa640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe1006a050 0x7fbe1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.725+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fbe18088770 con 0x7fbe2406bc80 2023-10-19T10:52:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.725+0000 7fbe28f93640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe1006a050 0x7fbe1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:35.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.726+0000 7fbe28f93640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe1006a050 0x7fbe1006c510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c036000 tx=0x7fbe1c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:35.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.729+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fbe18053cb0 con 0x7fbe2406bc80 2023-10-19T10:52:35.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.906+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe240ce070 con 0x7fbe2406bc80 2023-10-19T10:52:35.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.907+0000 7fbe217fa640 1 -- 172.21.15.50:0/3619406290 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe180572d0 con 0x7fbe2406bc80 2023-10-19T10:52:35.910 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:52:35.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe1006a050 msgr2=0x7fbe1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe1006a050 0x7fbe1006c510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c036000 tx=0x7fbe1c02e000 comp rx=0 tx=0).stop 2023-10-19T10:52:35.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 msgr2=0x7fbe2406c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:35.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 0x7fbe2406c0e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e850 tx=0x7fbe1800ed10 comp rx=0 tx=0).stop 2023-10-19T10:52:35.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 shutdown_connections 2023-10-19T10:52:35.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fbe1006a050 0x7fbe1006c510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe2415cf50 0x7fbe2415d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2406bc80 0x7fbe2406c0e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 --2- 172.21.15.50:0/3619406290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe0c003970 0x7fbe2415b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:35.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 >> 172.21.15.50:0/3619406290 conn(0x7fbe2409b960 msgr2=0x7fbe2415e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:35.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 shutdown_connections 2023-10-19T10:52:35.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:35.909+0000 7fbe2b1f7640 1 -- 172.21.15.50:0/3619406290 wait complete. 2023-10-19T10:52:35.924 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:52:36.033 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.030+0000 7f751ddd0640 1 Processor -- start 2023-10-19T10:52:36.033 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.030+0000 7f751ddd0640 1 -- start start 2023-10-19T10:52:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.030+0000 7f751ddd0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f75180d2db0 0x7f751815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.030+0000 7f751ddd0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751806aa30 con 0x7f75180d0e30 2023-10-19T10:52:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.030+0000 7f751ddd0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751806b2d0 con 0x7f751815c570 2023-10-19T10:52:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.030+0000 7f751ddd0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751815ce90 con 0x7f75180d2db0 2023-10-19T10:52:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f7516ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f75180d0e30 0x7f75180d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:51534/0 (socket says 172.21.15.50:51534) 2023-10-19T10:52:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f7516ffd640 1 -- 172.21.15.50:0/913223621 learned_addr learned my addr 172.21.15.50:0/913223621 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f7517fff640 1 --2- 172.21.15.50:0/913223621 >> v2:172.21.15.183:3300/0 conn(0x7f75180d2db0 0x7f751815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480220557 0 0) 0x7f751806aa30 con 0x7f75180d0e30 2023-10-19T10:52:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7500003520 con 0x7f75180d0e30 2023-10-19T10:52:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1678754594 0 0) 0x7f7500003520 con 0x7f75180d0e30 2023-10-19T10:52:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 >> v2:172.21.15.183:3300/0 conn(0x7f75180d2db0 msgr2=0x7f751815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 --2- 172.21.15.50:0/913223621 >> v2:172.21.15.183:3300/0 conn(0x7f75180d2db0 0x7f751815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 >> v1:172.21.15.183:6789/0 conn(0x7f751815c570 legacy=0x7f751806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180d1270 con 0x7f75180d0e30 2023-10-19T10:52:36.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f750c001670 con 0x7f75180d0e30 2023-10-19T10:52:36.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 >> v1:172.21.15.50:6789/0 conn(0x7f75180d0e30 legacy=0x7f75180d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7500003970 0x7f7500003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75000047c0 0x7f7500004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7500005180 0x7f750000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751815ce90 con 0x7f7500005180 2023-10-19T10:52:36.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750000c120 con 0x7f75000047c0 2023-10-19T10:52:36.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750000c2e0 con 0x7f7500003970 2023-10-19T10:52:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f7517fff640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7500005180 0x7f750000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f75177fe640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7500003970 0x7f7500003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.031+0000 7f7516ffd640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75000047c0 0x7f7500004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 -- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7500003970 msgr2=0x7f7500003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7500003970 0x7f7500003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 -- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75000047c0 msgr2=0x7f7500004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75000047c0 0x7f7500004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 -- 172.21.15.50:0/913223621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500003520 con 0x7f7500005180 2023-10-19T10:52:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 -- 172.21.15.50:0/913223621 reap_dead start 2023-10-19T10:52:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.032+0000 7f7517fff640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7500005180 0x7f750000b980 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7510009f90 tx=0x7f75000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b8af68b328ec81 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751002cc20 con 0x7f7500005180 2023-10-19T10:52:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751000f040 con 0x7f7500005180 2023-10-19T10:52:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f75167fc640 1 -- 172.21.15.50:0/913223621 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510003990 con 0x7f7500005180 2023-10-19T10:52:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 -- 172.21.15.50:0/913223621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7500005180 msgr2=0x7f750000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7500005180 0x7f750000b980 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7510009f90 tx=0x7f75000057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 -- 172.21.15.50:0/913223621 shutdown_connections 2023-10-19T10:52:36.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 --2- 172.21.15.50:0/913223621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7500005180 0x7f750000b980 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 -- 172.21.15.50:0/913223621 >> 172.21.15.50:0/913223621 conn(0x7f7518089ff0 msgr2=0x7f751809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:36.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 -- 172.21.15.50:0/913223621 shutdown_connections 2023-10-19T10:52:36.044 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.033+0000 7f751ddd0640 1 -- 172.21.15.50:0/913223621 wait complete. 2023-10-19T10:52:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.034+0000 7f751ddd0640 1 Processor -- start 2023-10-19T10:52:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 -- start start 2023-10-19T10:52:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7500005180 0x7f75181f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75180b7a20 0x7f751815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 0x7f751815ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75180b54d0 con 0x7f75180b7ea0 2023-10-19T10:52:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751815f620 con 0x7f7500005180 2023-10-19T10:52:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f751ddd0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751815f900 con 0x7f75180b7a20 2023-10-19T10:52:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f7517fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 0x7f751815ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f7517fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 0x7f751815ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48888/0 (socket says 172.21.15.50:48888) 2023-10-19T10:52:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.035+0000 7f7517fff640 1 -- 172.21.15.50:0/102521046 learned_addr learned my addr 172.21.15.50:0/102521046 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f7517fff640 1 -- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75180b7a20 msgr2=0x7f751815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f7517fff640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75180b7a20 0x7f751815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f7517fff640 1 -- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7500005180 msgr2=0x7f75181f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f75177fe640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7500005180 0x7f75181f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f7517fff640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7500005180 0x7f75181f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f7517fff640 1 -- 172.21.15.50:0/102521046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751815fbe0 con 0x7f75180b7ea0 2023-10-19T10:52:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f7517fff640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 0x7f751815ee00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f751002c690 tx=0x7f7510009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510004230 con 0x7f75180b7ea0 2023-10-19T10:52:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7510004390 con 0x7f75180b7ea0 2023-10-19T10:52:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751815f340 con 0x7f75180b7ea0 2023-10-19T10:52:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510004740 con 0x7f75180b7ea0 2023-10-19T10:52:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.036+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75181596c0 con 0x7f75180b7ea0 2023-10-19T10:52:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.037+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75180b52a0 con 0x7f75180b7ea0 2023-10-19T10:52:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.037+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f751003f470 con 0x7f75180b7ea0 2023-10-19T10:52:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.038+0000 7f74fffff640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.038+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f75100ab090 con 0x7f75180b7ea0 2023-10-19T10:52:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.038+0000 7f75177fe640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.039+0000 7f75177fe640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7508000c00 tx=0x7f7508009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.042+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f75180b52a0 con 0x7f75180b7ea0 2023-10-19T10:52:36.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.230+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f75180b52a0 con 0x7f75180b7ea0 2023-10-19T10:52:36.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.230+0000 7f74fffff640 1 -- 172.21.15.50:0/102521046 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v55) v1 ==== 80+0+56 (secure 0 0 0) 0x7f75180b52a0 con 0x7f75180b7ea0 2023-10-19T10:52:36.234 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.1:0/216420809 2023-10-19T11:46:25.724828+0000 2023-10-19T10:52:36.234 INFO:teuthology.orchestra.run.smithi050.stderr:listed 1 entries 2023-10-19T10:52:36.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 msgr2=0x7f750406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.235 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7508000c00 tx=0x7f7508009040 comp rx=0 tx=0).stop 2023-10-19T10:52:36.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 msgr2=0x7f751815ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 0x7f751815ee00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f751002c690 tx=0x7f7510009750 comp rx=0 tx=0).stop 2023-10-19T10:52:36.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 shutdown_connections 2023-10-19T10:52:36.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.236 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75180b7ea0 0x7f751815ee00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75180b7a20 0x7f751815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 --2- 172.21.15.50:0/102521046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7500005180 0x7f75181f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 >> 172.21.15.50:0/102521046 conn(0x7f7518089ff0 msgr2=0x7f75180b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:36.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 shutdown_connections 2023-10-19T10:52:36.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.232+0000 7f751ddd0640 1 -- 172.21.15.50:0/102521046 wait complete. 2023-10-19T10:52:36.246 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-10-19T10:52:36.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.247 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:36.278 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:52:36.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.279 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-19T10:52:36.334 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-19T10:52:36.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.478+0000 7f7922fa0640 1 Processor -- start 2023-10-19T10:52:36.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.478+0000 7f7922fa0640 1 -- start start 2023-10-19T10:52:36.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f7922fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 0x7f791c167760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f7922fa0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c15ee00 con 0x7f791c0b1f30 2023-10-19T10:52:36.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f7922fa0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c15d6e0 con 0x7f791c160340 2023-10-19T10:52:36.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f7922fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c15f030 con 0x7f791c165370 2023-10-19T10:52:36.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f7920d3c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f791c0b1f30 0x7f791c0b2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:51548/0 (socket says 172.21.15.50:51548) 2023-10-19T10:52:36.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f7920d3c640 1 -- 172.21.15.50:0/3988645978 learned_addr learned my addr 172.21.15.50:0/3988645978 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:36.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f792153d640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 0x7f791c167760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817361046 0 0) 0x7f791c15ee00 con 0x7f791c0b1f30 2023-10-19T10:52:36.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.479+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7904003520 con 0x7f791c0b1f30 2023-10-19T10:52:36.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527353090 0 0) 0x7f7904003520 con 0x7f791c0b1f30 2023-10-19T10:52:36.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 msgr2=0x7f791c167760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:36.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 0x7f791c167760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 >> v1:172.21.15.183:6789/0 conn(0x7f791c160340 legacy=0x7f791c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c15cdb0 con 0x7f791c0b1f30 2023-10-19T10:52:36.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f79140019f0 con 0x7f791c0b1f30 2023-10-19T10:52:36.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 >> v1:172.21.15.50:6789/0 conn(0x7f791c0b1f30 legacy=0x7f791c0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 0x7f79040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7904003ee0 0x7f79040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7904004d00 0x7f7904005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c15f030 con 0x7f7904004d00 2023-10-19T10:52:36.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790400ba90 con 0x7f7904003ee0 2023-10-19T10:52:36.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790400bc50 con 0x7f791c165370 2023-10-19T10:52:36.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f792153d640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7904004d00 0x7f7904005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f791bfff640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7904003ee0 0x7f79040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f7920d3c640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 0x7f79040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f792153d640 1 -- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 msgr2=0x7f79040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:36.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f792153d640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c165370 0x7f79040039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f792153d640 1 -- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7904003ee0 msgr2=0x7f79040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:36.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f792153d640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7904003ee0 0x7f79040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.480+0000 7f792153d640 1 -- 172.21.15.50:0/3988645978 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904003520 con 0x7f7904004d00 2023-10-19T10:52:36.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.481+0000 7f792153d640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7904004d00 0x7f7904005180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ac0 tx=0x7f79040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fc34660bbac064 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:36.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.481+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c02c6b0 con 0x7f7904004d00 2023-10-19T10:52:36.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.481+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c02c810 con 0x7f7904004d00 2023-10-19T10:52:36.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.481+0000 7f791b7fe640 1 -- 172.21.15.50:0/3988645978 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c02cba0 con 0x7f7904004d00 2023-10-19T10:52:36.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f7922fa0640 1 -- 172.21.15.50:0/3988645978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7904004d00 msgr2=0x7f7904005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f7922fa0640 1 --2- 172.21.15.50:0/3988645978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7904004d00 0x7f7904005180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ac0 tx=0x7f79040057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f792153d640 1 -- 172.21.15.50:0/3988645978 reap_dead start 2023-10-19T10:52:36.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f7922fa0640 1 -- 172.21.15.50:0/3988645978 shutdown_connections 2023-10-19T10:52:36.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f7922fa0640 1 -- 172.21.15.50:0/3988645978 >> 172.21.15.50:0/3988645978 conn(0x7f791c0ccf30 msgr2=0x7f791c0cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:36.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f7922fa0640 1 -- 172.21.15.50:0/3988645978 shutdown_connections 2023-10-19T10:52:36.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.482+0000 7f7922fa0640 1 -- 172.21.15.50:0/3988645978 wait complete. 2023-10-19T10:52:36.499 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.483+0000 7f7922fa0640 1 Processor -- start 2023-10-19T10:52:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.483+0000 7f7922fa0640 1 -- start start 2023-10-19T10:52:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7922fa0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c157a20 0x7f791c0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7922fa0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791c0c6980 0x7f791c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7922fa0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 0x7f791c0ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.500 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7922fa0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c5c20 con 0x7f791c0c6e00 2023-10-19T10:52:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7922fa0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c5030 con 0x7f791c0c6980 2023-10-19T10:52:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7922fa0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c6580 con 0x7f791c157a20 2023-10-19T10:52:36.501 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 0x7f791c0ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 0x7f791c0ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48916/0 (socket says 172.21.15.50:48916) 2023-10-19T10:52:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 -- 172.21.15.50:0/4283228254 learned_addr learned my addr 172.21.15.50:0/4283228254 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f7920d3c640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c157a20 0x7f791c0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f791bfff640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791c0c6980 0x7f791c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.502 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 -- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c157a20 msgr2=0x7f791c0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c157a20 0x7f791c0c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 -- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791c0c6980 msgr2=0x7f791c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791c0c6980 0x7f791c0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 -- 172.21.15.50:0/4283228254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0c5280 con 0x7f791c0c6e00 2023-10-19T10:52:36.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.484+0000 7f792153d640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 0x7f791c0ce5a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ac0 tx=0x7f790c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:36.503 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.485+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c02c6b0 con 0x7f791c0c6e00 2023-10-19T10:52:36.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.485+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c003810 con 0x7f791c0c6e00 2023-10-19T10:52:36.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.485+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c0c5560 con 0x7f791c0c6e00 2023-10-19T10:52:36.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.485+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c003b80 con 0x7f791c0c6e00 2023-10-19T10:52:36.504 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.485+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c157e00 con 0x7f791c0c6e00 2023-10-19T10:52:36.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.486+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c157f90 con 0x7f791c0c6e00 2023-10-19T10:52:36.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.486+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f790c003ce0 con 0x7f791c0c6e00 2023-10-19T10:52:36.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.487+0000 7f79197fa640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f790806a050 0x7f790806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:36.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.487+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f790c0a9dc0 con 0x7f791c0c6e00 2023-10-19T10:52:36.505 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.487+0000 7f7920d3c640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f790806a050 0x7f790806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:36.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.487+0000 7f7920d3c640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f790806a050 0x7f790806c510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7914001460 tx=0x7f7914006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:36.506 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.491+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f790c075620 con 0x7f791c0c6e00 2023-10-19T10:52:36.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.704+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f791c073990 con 0x7f791c0c6e00 2023-10-19T10:52:36.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.704+0000 7f79197fa640 1 -- 172.21.15.50:0/4283228254 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f790c0044e0 con 0x7f791c0c6e00 2023-10-19T10:52:36.707 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-19T10:52:36.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f790806a050 msgr2=0x7f790806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f790806a050 0x7f790806c510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7914001460 tx=0x7f7914006040 comp rx=0 tx=0).stop 2023-10-19T10:52:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 msgr2=0x7f791c0ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 0x7f791c0ce5a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ac0 tx=0x7f790c004730 comp rx=0 tx=0).stop 2023-10-19T10:52:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 shutdown_connections 2023-10-19T10:52:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f790806a050 0x7f790806c510 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.710 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f791c0c6e00 0x7f791c0ce5a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791c0c6980 0x7f791c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 --2- 172.21.15.50:0/4283228254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f791c157a20 0x7f791c0c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:36.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 >> 172.21.15.50:0/4283228254 conn(0x7f791c0ccf30 msgr2=0x7f791c0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:36.711 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 shutdown_connections 2023-10-19T10:52:36.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:36.706+0000 7f7922fa0640 1 -- 172.21.15.50:0/4283228254 wait complete. 2023-10-19T10:52:36.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-19T10:52:36.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-19T10:52:36.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2023-10-19T10:52:36.721 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-19T10:52:36.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:36.722 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-19T10:52:36.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-19T10:52:36.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-19T10:52:36.723 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:52:36.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.724 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2023-10-19T10:52:36.740 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-19T10:52:36.740 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-19T10:52:36.740 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2023-10-19T10:52:36.740 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:52:36.741 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2023-10-19T10:52:36.741 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:52:36.741 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:52:36.741 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:52:36.741 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2023-10-19T10:52:36.742 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:52:36.742 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:52:36.742 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2023-10-19T10:52:36.742 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-19T10:52:36.742 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:52:36.743 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2023-10-19T10:52:36.743 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-19T10:52:36.743 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41744sec preferred_lft 41744sec 2023-10-19T10:52:36.743 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2023-10-19T10:52:36.743 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:52:36.744 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-19T10:52:36.744 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2023-10-19T10:52:36.744 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2023-10-19T10:52:36.744 INFO:teuthology.orchestra.run.smithi050.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-19T10:52:36.744 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c2:88:d7:94:a0:7a brd ff:ff:ff:ff:ff:ff 2023-10-19T10:52:36.744 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-19T10:52:36.745 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:52:36.745 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a497:7bff:fe82:fcde/64 scope link 2023-10-19T10:52:36.745 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:52:36.745 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-19T10:52:36.745 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether fa:5b:82:63:89:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-19T10:52:36.746 INFO:teuthology.orchestra.run.smithi050.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-19T10:52:36.746 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether c2:88:d7:94:a0:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-19T10:52:36.746 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::c088:d7ff:fe94:a07a/64 scope link 2023-10-19T10:52:36.746 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2023-10-19T10:52:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.747 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-19T10:52:36.809 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-10-19T10:52:36.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.811 DEBUG:teuthology.orchestra.run.smithi050:> route 2023-10-19T10:52:36.867 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2023-10-19T10:52:36.867 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-19T10:52:36.867 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-19T10:52:36.868 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-19T10:52:36.868 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-19T10:52:36.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.870 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec bash -c ' 2023-10-19T10:52:36.870 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2023-10-19T10:52:36.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-19T10:52:36.871 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-19T10:52:36.871 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-19T10:52:36.872 DEBUG:teuthology.orchestra.run.smithi050:> ') 2023-10-19T10:52:36.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.974 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:52:36.989 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-19T10:52:36.990 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-19T10:52:36.990 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:52:36.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:36.991 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2023-10-19T10:52:37.060 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-19T10:52:37.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:52:37.061 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:37.116 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-19T10:52:37.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-19T10:52:37.117 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:37.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:52:37.243 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:52:37.244 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:52:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:52:37.315 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:52:37.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:37.316 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:52:37.370 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-19T10:52:37.371 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-19T10:52:37.413 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2023-10-19T10:52:37.445 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-19T10:52:37.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-19T10:52:37.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:2023-10-19T10:52:37.504+0000 7f9177237480 -1 init, newargv = 0x5612b90e5180 newargc=15 2023-10-19T10:52:37.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[109568]: starting ceph client 2023-10-19T10:52:37.512 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[109568]: starting fuse 2023-10-19T10:52:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-19T10:52:37.518 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-19T10:52:37.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:37.519 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2023-10-19T10:52:37.575 INFO:teuthology.orchestra.run.smithi050.stdout:41 2023-10-19T10:52:37.575 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-19T10:52:37.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2023-10-19T10:52:37.576 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2023-10-19T10:52:37.577 DEBUG:teuthology.orchestra.run.smithi050:> import re 2023-10-19T10:52:37.577 DEBUG:teuthology.orchestra.run.smithi050:> import os 2023-10-19T10:52:37.577 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2023-10-19T10:52:37.578 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:52:37.578 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2023-10-19T10:52:37.579 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-19T10:52:37.579 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2023-10-19T10:52:37.579 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-19T10:52:37.580 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:52:37.580 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2023-10-19T10:52:37.581 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2023-10-19T10:52:37.581 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2023-10-19T10:52:37.581 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2023-10-19T10:52:37.582 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:52:37.582 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2023-10-19T10:52:37.583 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-19T10:52:37.583 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-19T10:52:37.584 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-19T10:52:37.584 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2023-10-19T10:52:37.584 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2023-10-19T10:52:37.585 DEBUG:teuthology.orchestra.run.smithi050:> return f 2023-10-19T10:52:37.585 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-19T10:52:37.586 DEBUG:teuthology.orchestra.run.smithi050:> 2023-10-19T10:52:37.586 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.0")) 2023-10-19T10:52:37.587 DEBUG:teuthology.orchestra.run.smithi050:> ' 2023-10-19T10:52:37.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.0.109568.asok 2023-10-19T10:52:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-19T10:52:37.709 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109568.asok 2023-10-19T10:52:37.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:37.710 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109568.asok status 2023-10-19T10:52:37.851 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2023-10-19T10:52:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-19T10:52:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-19T10:52:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "0", 2023-10-19T10:52:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2023-10-19T10:52:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-19T10:52:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "109568", 2023-10-19T10:52:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2023-10-19T10:52:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:52:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2023-10-19T10:52:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2023-10-19T10:52:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 6255, 2023-10-19T10:52:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2023-10-19T10:52:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2023-10-19T10:52:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2023-10-19T10:52:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 6255 2023-10-19T10:52:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:52:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:52:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:52:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:52:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 3842335686 2023-10-19T10:52:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: } 2023-10-19T10:52:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:52:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2023-10-19T10:52:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2023-10-19T10:52:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2023-10-19T10:52:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 3842335686 2023-10-19T10:52:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2023-10-19T10:52:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.6255 192.168.0.1:0/3842335686", 2023-10-19T10:52:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/3842335686", 2023-10-19T10:52:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2023-10-19T10:52:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 17, 2023-10-19T10:52:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 55, 2023-10-19T10:52:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2023-10-19T10:52:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2023-10-19T10:52:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2023-10-19T10:52:37.859 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:37.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:37.860 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2023-10-19T10:52:37.861 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:37.876 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2023-10-19T10:52:37.876 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:37.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:37.877 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:37.947 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:52:37.948 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:38.003 DEBUG:tasks.cephfs.mount:Creating test file 2023-10-19 10:52:38.003196 0 2023-10-19T10:52:38.003 DEBUG:teuthology.orchestra.run.smithi050:> touch '/home/ubuntu/cephtest/mnt.0/2023-10-19 10:52:38.003196 0' 2023-10-19T10:52:38.062 DEBUG:tasks.cephfs.mount:Deleting test file 2023-10-19 10:52:38.003196 0 2023-10-19T10:52:38.062 DEBUG:teuthology.orchestra.run.smithi050:> rm -f '/home/ubuntu/cephtest/mnt.0/2023-10-19 10:52:38.003196 0' 2023-10-19T10:52:38.118 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-19T10:52:38.119 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-19T10:52:38.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-19T10:52:38.252 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-19T10:52:38.253 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:52:38.253 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:38.269 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:38.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.411+0000 7f1fc2bd2640 1 Processor -- start 2023-10-19T10:52:38.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.411+0000 7f1fc2bd2640 1 -- start start 2023-10-19T10:52:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fc2bd2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fbc15cd90 0x7f1fbc06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fc2bd2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc0ca480 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fc2bd2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc159990 con 0x7f1fbc159280 2023-10-19T10:52:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fc2bd2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc15d500 con 0x7f1fbc15cd90 2023-10-19T10:52:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fc116f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1fbc15cd90 0x7f1fbc06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fbbfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1fbc0cd7d0 0x7f1fbc159e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50026/0 (socket says 172.21.15.50:50026) 2023-10-19T10:52:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.412+0000 7f1fbbfff640 1 -- 172.21.15.50:0/435012712 learned_addr learned my addr 172.21.15.50:0/435012712 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.413+0000 7f1fc116f640 1 -- 172.21.15.50:0/435012712 >> v1:172.21.15.183:6789/0 conn(0x7f1fbc159280 legacy=0x7f1fbc0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:38.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.413+0000 7f1fc116f640 1 -- 172.21.15.50:0/435012712 >> v1:172.21.15.50:6789/0 conn(0x7f1fbc0cd7d0 legacy=0x7f1fbc159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.413+0000 7f1fbb7fe640 1 -- 172.21.15.50:0/435012712 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481191598 0 0) 0x7f1fbc0ca480 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.413+0000 7f1fc116f640 1 -- 172.21.15.50:0/435012712 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc15d7e0 con 0x7f1fbc15cd90 2023-10-19T10:52:38.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.413+0000 7f1fc116f640 1 --2- 172.21.15.50:0/435012712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc15cd90 0x7f1fbc06b4a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097a0 tx=0x7f1fac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf30d791e2a32b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:38.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fbb7fe640 1 -- 172.21.15.50:0/435012712 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac02e070 con 0x7f1fbc15cd90 2023-10-19T10:52:38.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fbb7fe640 1 -- 172.21.15.50:0/435012712 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac0315a0 con 0x7f1fbc15cd90 2023-10-19T10:52:38.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fbb7fe640 1 -- 172.21.15.50:0/435012712 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac031930 con 0x7f1fbc15cd90 2023-10-19T10:52:38.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/435012712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc15cd90 msgr2=0x7f1fbc06b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/435012712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc15cd90 0x7f1fbc06b4a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097a0 tx=0x7f1fac02b840 comp rx=0 tx=0).stop 2023-10-19T10:52:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/435012712 shutdown_connections 2023-10-19T10:52:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/435012712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc15cd90 0x7f1fbc06b4a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/435012712 >> 172.21.15.50:0/435012712 conn(0x7f1fbc069e90 msgr2=0x7f1fbc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:38.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/435012712 shutdown_connections 2023-10-19T10:52:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.414+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/435012712 wait complete. 2023-10-19T10:52:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.415+0000 7f1fc2bd2640 1 Processor -- start 2023-10-19T10:52:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.415+0000 7f1fc2bd2640 1 -- start start 2023-10-19T10:52:38.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc2bd2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 0x7f1fbc14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc2bd2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fbc159280 0x7f1fbc0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc2bd2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fbc15cd90 0x7f1fbc152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc2bd2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc14f7c0 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc2bd2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc1f3930 con 0x7f1fbc159280 2023-10-19T10:52:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc2bd2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc1f3c10 con 0x7f1fbc15cd90 2023-10-19T10:52:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc096e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 0x7f1fbc14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc096e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 0x7f1fbc14ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55948/0 (socket says 172.21.15.50:55948) 2023-10-19T10:52:38.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc096e640 1 -- 172.21.15.50:0/3932516486 learned_addr learned my addr 172.21.15.50:0/3932516486 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:38.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fbbfff640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fbc159280 0x7f1fbc0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.416+0000 7f1fc116f640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fbc15cd90 0x7f1fbc152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc096e640 1 -- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fbc15cd90 msgr2=0x7f1fbc152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc096e640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fbc15cd90 0x7f1fbc152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc096e640 1 -- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fbc159280 msgr2=0x7f1fbc0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.430 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc096e640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fbc159280 0x7f1fbc0c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc096e640 1 -- 172.21.15.50:0/3932516486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009400 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc096e640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 0x7f1fbc14ea70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c7c0 tx=0x7f1fb400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:38.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400aa50 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb400abb0 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.431 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc1f3ef0 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4012420 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.417+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc14fd70 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.418+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc0c19b0 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.418+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1fb4012580 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.432 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.419+0000 7f1fb97fa640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fa806a050 0x7f1fa806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.419+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1fb4088640 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.419+0000 7f1fbbfff640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fa806a050 0x7f1fa806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.419+0000 7f1fbbfff640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fa806a050 0x7f1fa806c510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc06c6f0 tx=0x7f1fb0007b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:38.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.423+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1fb4053c10 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.600+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fbc073990 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.602+0000 7f1fb97fa640 1 -- 172.21.15.50:0/3932516486 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18701 (secure 0 0 0) 0x7f1fb4057230 con 0x7f1fbc0cd7d0 2023-10-19T10:52:38.605 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:38.605 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:46:26.319171+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:38.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.603+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fa806a050 msgr2=0x7f1fa806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.603+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fa806a050 0x7f1fa806c510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc06c6f0 tx=0x7f1fb0007b90 comp rx=0 tx=0).stop 2023-10-19T10:52:38.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.603+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 msgr2=0x7f1fbc14ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.603+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 0x7f1fbc14ea70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c7c0 tx=0x7f1fb400cc80 comp rx=0 tx=0).stop 2023-10-19T10:52:38.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.603+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 shutdown_connections 2023-10-19T10:52:38.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.603+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1fa806a050 0x7f1fa806c510 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.604+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fbc15cd90 0x7f1fbc152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.604+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fbc159280 0x7f1fbc0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.604+0000 7f1fc2bd2640 1 --2- 172.21.15.50:0/3932516486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fbc0cd7d0 0x7f1fbc14ea70 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.604+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 >> 172.21.15.50:0/3932516486 conn(0x7f1fbc069e90 msgr2=0x7f1fbc070680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:38.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.604+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 shutdown_connections 2023-10-19T10:52:38.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.604+0000 7f1fc2bd2640 1 -- 172.21.15.50:0/3932516486 wait complete. 2023-10-19T10:52:38.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:38:26.398890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:38:26.910608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:52:38.619 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:52:38.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.725+0000 7f63e61ce640 1 Processor -- start 2023-10-19T10:52:38.728 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.725+0000 7f63e61ce640 1 -- start start 2023-10-19T10:52:38.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.725+0000 7f63e61ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e00b4070 0x7f63e00b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.725+0000 7f63e61ce640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f63e015cdc0 0x7f63e00cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.725+0000 7f63e61ce640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e00d3480 con 0x7f63e015c960 2023-10-19T10:52:38.730 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63e61ce640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e00b4990 con 0x7f63e015cdc0 2023-10-19T10:52:38.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63e61ce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e015d280 con 0x7f63e00b4070 2023-10-19T10:52:38.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63dffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f63e015cdc0 0x7f63e00cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63deffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f63e015c960 0x7f63e00d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50038/0 (socket says 172.21.15.50:50038) 2023-10-19T10:52:38.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63deffd640 1 -- 172.21.15.50:0/422107459 learned_addr learned my addr 172.21.15.50:0/422107459 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:38.731 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63df7fe640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e00b4070 0x7f63e00b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63dffff640 1 -- 172.21.15.50:0/422107459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e00b4070 msgr2=0x7f63e00b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:38.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63de7fc640 1 -- 172.21.15.50:0/422107459 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210162107 0 0) 0x7f63e00d3480 con 0x7f63e015c960 2023-10-19T10:52:38.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63dffff640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e00b4070 0x7f63e00b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63dffff640 1 -- 172.21.15.50:0/422107459 >> v1:172.21.15.50:6789/0 conn(0x7f63e015c960 legacy=0x7f63e00d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63dffff640 1 -- 172.21.15.50:0/422107459 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e015d560 con 0x7f63e015cdc0 2023-10-19T10:52:38.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.726+0000 7f63df7fe640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e00b4070 0x7f63e00b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:38.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63dffff640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015cdc0 0x7f63e00cd4b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009d90 tx=0x7f63d802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6d10c2531d7e34 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:38.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63de7fc640 1 -- 172.21.15.50:0/422107459 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d802cc50 con 0x7f63e015cdc0 2023-10-19T10:52:38.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63de7fc640 1 -- 172.21.15.50:0/422107459 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d802cdb0 con 0x7f63e015cdc0 2023-10-19T10:52:38.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63de7fc640 1 -- 172.21.15.50:0/422107459 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8034660 con 0x7f63e015cdc0 2023-10-19T10:52:38.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63e61ce640 1 -- 172.21.15.50:0/422107459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015cdc0 msgr2=0x7f63e00cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63e61ce640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015cdc0 0x7f63e00cd4b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009d90 tx=0x7f63d802bea0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63e61ce640 1 -- 172.21.15.50:0/422107459 shutdown_connections 2023-10-19T10:52:38.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63e61ce640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015cdc0 0x7f63e00cd4b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63e61ce640 1 --2- 172.21.15.50:0/422107459 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e00b4070 0x7f63e00b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.727+0000 7f63e61ce640 1 -- 172.21.15.50:0/422107459 >> 172.21.15.50:0/422107459 conn(0x7f63e009b970 msgr2=0x7f63e00d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:38.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.728+0000 7f63e61ce640 1 -- 172.21.15.50:0/422107459 shutdown_connections 2023-10-19T10:52:38.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.728+0000 7f63e61ce640 1 -- 172.21.15.50:0/422107459 wait complete. 2023-10-19T10:52:38.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.729+0000 7f63e61ce640 1 Processor -- start 2023-10-19T10:52:38.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.729+0000 7f63e61ce640 1 -- start start 2023-10-19T10:52:38.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.729+0000 7f63e61ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e00b4070 0x7f63e01ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 0x7f63e01f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e015cdc0 0x7f63e00b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e01f30d0 con 0x7f63e015c960 2023-10-19T10:52:38.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00b0610 con 0x7f63e00b4070 2023-10-19T10:52:38.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e01efad0 con 0x7f63e015cdc0 2023-10-19T10:52:38.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 0x7f63e01f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 0x7f63e01f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55966/0 (socket says 172.21.15.50:55966) 2023-10-19T10:52:38.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 -- 172.21.15.50:0/2677833168 learned_addr learned my addr 172.21.15.50:0/2677833168 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:38.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63df7fe640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e00b4070 0x7f63e01ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63dffff640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e015cdc0 0x7f63e00b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 -- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e015cdc0 msgr2=0x7f63e00b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e015cdc0 0x7f63e00b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 -- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e00b4070 msgr2=0x7f63e01ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e00b4070 0x7f63e01ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 -- 172.21.15.50:0/2677833168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8009a70 con 0x7f63e015c960 2023-10-19T10:52:38.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63deffd640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 0x7f63e01f20e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c4d0 tx=0x7f63d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:38.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4005ce0 con 0x7f63e015c960 2023-10-19T10:52:38.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d4005e60 con 0x7f63e015c960 2023-10-19T10:52:38.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e01f2b10 con 0x7f63e015c960 2023-10-19T10:52:38.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d40072b0 con 0x7f63e015c960 2023-10-19T10:52:38.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.730+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e00b02a0 con 0x7f63e015c960 2023-10-19T10:52:38.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.731+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e00b07a0 con 0x7f63e015c960 2023-10-19T10:52:38.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.732+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f63d4007410 con 0x7f63e015c960 2023-10-19T10:52:38.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.732+0000 7f63c7fff640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f63cc06a050 0x7f63cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:38.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.732+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f63d4087cd0 con 0x7f63e015c960 2023-10-19T10:52:38.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.732+0000 7f63df7fe640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f63cc06a050 0x7f63cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:38.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.733+0000 7f63df7fe640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f63cc06a050 0x7f63cc06c510 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63d0006fd0 tx=0x7f63d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:38.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.737+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f63d4053220 con 0x7f63e015c960 2023-10-19T10:52:38.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.910+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63e004e420 con 0x7f63e015c960 2023-10-19T10:52:38.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.910+0000 7f63c7fff640 1 -- 172.21.15.50:0/2677833168 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63d4056840 con 0x7f63e015c960 2023-10-19T10:52:38.914 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:52:38.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f63cc06a050 msgr2=0x7f63cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f63cc06a050 0x7f63cc06c510 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63d0006fd0 tx=0x7f63d0008040 comp rx=0 tx=0).stop 2023-10-19T10:52:38.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 msgr2=0x7f63e01f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:38.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 0x7f63e01f20e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c4d0 tx=0x7f63d400c990 comp rx=0 tx=0).stop 2023-10-19T10:52:38.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 shutdown_connections 2023-10-19T10:52:38.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f63cc06a050 0x7f63cc06c510 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e015cdc0 0x7f63e00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63e015c960 0x7f63e01f20e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 --2- 172.21.15.50:0/2677833168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e00b4070 0x7f63e01ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:38.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 >> 172.21.15.50:0/2677833168 conn(0x7f63e009b970 msgr2=0x7f63e00cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:38.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 shutdown_connections 2023-10-19T10:52:38.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:38.912+0000 7f63e61ce640 1 -- 172.21.15.50:0/2677833168 wait complete. 2023-10-19T10:52:38.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:52:39.034 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.031+0000 7f9993900640 1 Processor -- start 2023-10-19T10:52:39.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.031+0000 7f9993900640 1 -- start start 2023-10-19T10:52:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f9993900640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c0cd460 0x7f998c06a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f9993900640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f998c160240 0x7f998c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f9993900640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c070780 con 0x7f998c15c660 2023-10-19T10:52:39.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f9993900640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c0cca10 con 0x7f998c160240 2023-10-19T10:52:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f9993900640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c06a750 con 0x7f998c0cd460 2023-10-19T10:52:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f9991e9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f998c160240 0x7f998c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f999169c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f998c15c660 0x7f998c0ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50048/0 (socket says 172.21.15.50:50048) 2023-10-19T10:52:39.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.032+0000 7f999169c640 1 -- 172.21.15.50:0/898858189 learned_addr learned my addr 172.21.15.50:0/898858189 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.033+0000 7f9990e9b640 1 --2- 172.21.15.50:0/898858189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c0cd460 0x7f998c06a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.033+0000 7f9991e9d640 1 -- 172.21.15.50:0/898858189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c0cd460 msgr2=0x7f998c06a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.033+0000 7f9991e9d640 1 --2- 172.21.15.50:0/898858189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c0cd460 0x7f998c06a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.040 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.033+0000 7f9991e9d640 1 -- 172.21.15.50:0/898858189 >> v1:172.21.15.50:6789/0 conn(0x7f998c15c660 legacy=0x7f998c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.033+0000 7f998bfff640 1 -- 172.21.15.50:0/898858189 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082767737 0 0) 0x7f998c070780 con 0x7f998c15c660 2023-10-19T10:52:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.033+0000 7f9991e9d640 1 -- 172.21.15.50:0/898858189 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c160720 con 0x7f998c160240 2023-10-19T10:52:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f9991e9d640 1 --2- 172.21.15.50:0/898858189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c160240 0x7f998c070c10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f997c009d90 tx=0x7f997c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726a92551bb1892d server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f998bfff640 1 -- 172.21.15.50:0/898858189 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c02cc50 con 0x7f998c160240 2023-10-19T10:52:39.041 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f998bfff640 1 -- 172.21.15.50:0/898858189 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c02cdb0 con 0x7f998c160240 2023-10-19T10:52:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f998bfff640 1 -- 172.21.15.50:0/898858189 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c034660 con 0x7f998c160240 2023-10-19T10:52:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f9993900640 1 -- 172.21.15.50:0/898858189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c160240 msgr2=0x7f998c070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f9993900640 1 --2- 172.21.15.50:0/898858189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c160240 0x7f998c070c10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f997c009d90 tx=0x7f997c02bea0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.034+0000 7f9993900640 1 -- 172.21.15.50:0/898858189 shutdown_connections 2023-10-19T10:52:39.042 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.035+0000 7f9993900640 1 --2- 172.21.15.50:0/898858189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c160240 0x7f998c070c10 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.035+0000 7f9993900640 1 --2- 172.21.15.50:0/898858189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c0cd460 0x7f998c06a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.035+0000 7f9993900640 1 -- 172.21.15.50:0/898858189 >> 172.21.15.50:0/898858189 conn(0x7f998c0cbc00 msgr2=0x7f998c0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.035+0000 7f9993900640 1 -- 172.21.15.50:0/898858189 shutdown_connections 2023-10-19T10:52:39.043 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.035+0000 7f9993900640 1 -- 172.21.15.50:0/898858189 wait complete. 2023-10-19T10:52:39.046 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.036+0000 7f9993900640 1 Processor -- start 2023-10-19T10:52:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.036+0000 7f9993900640 1 -- start start 2023-10-19T10:52:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 0x7f998c0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c15c660 0x7f998c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c160240 0x7f998c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.047 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0b2f20 con 0x7f998c0cd460 2023-10-19T10:52:39.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0b3180 con 0x7f998c15c660 2023-10-19T10:52:39.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0c4600 con 0x7f998c160240 2023-10-19T10:52:39.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 0x7f998c0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 0x7f998c0c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55986/0 (socket says 172.21.15.50:55986) 2023-10-19T10:52:39.048 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 -- 172.21.15.50:0/3712918626 learned_addr learned my addr 172.21.15.50:0/3712918626 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9990e9b640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c15c660 0x7f998c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9991e9d640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c160240 0x7f998c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 -- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c160240 msgr2=0x7f998c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c160240 0x7f998c0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 -- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c15c660 msgr2=0x7f998c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c15c660 0x7f998c0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 -- 172.21.15.50:0/3712918626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009a70 con 0x7f998c0cd460 2023-10-19T10:52:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9990e9b640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c15c660 0x7f998c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:39.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f999169c640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 0x7f998c0c3fd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f998400c430 tx=0x7f998400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840074e0 con 0x7f998c0cd460 2023-10-19T10:52:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9984007660 con 0x7f998c0cd460 2023-10-19T10:52:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c0c48e0 con 0x7f998c0cd460 2023-10-19T10:52:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.037+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984004a20 con 0x7f998c0cd460 2023-10-19T10:52:39.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.038+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c0c4ea0 con 0x7f998c0cd460 2023-10-19T10:52:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.039+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c0c50c0 con 0x7f998c0cd460 2023-10-19T10:52:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.039+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9984004b80 con 0x7f998c0cd460 2023-10-19T10:52:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.039+0000 7f9989ffb640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f997806a050 0x7f997806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.039+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f9984088560 con 0x7f998c0cd460 2023-10-19T10:52:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.039+0000 7f9990e9b640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f997806a050 0x7f997806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.040+0000 7f9990e9b640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f997806a050 0x7f997806c510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9980006fd0 tx=0x7f9980008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.044+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9984053aa0 con 0x7f998c0cd460 2023-10-19T10:52:39.220 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.217+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f998c04e420 con 0x7f998c0cd460 2023-10-19T10:52:39.221 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.217+0000 7f9989ffb640 1 -- 172.21.15.50:0/3712918626 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99840570c0 con 0x7f998c0cd460 2023-10-19T10:52:39.221 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:52:39.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f997806a050 msgr2=0x7f997806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f997806a050 0x7f997806c510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9980006fd0 tx=0x7f9980008040 comp rx=0 tx=0).stop 2023-10-19T10:52:39.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 msgr2=0x7f998c0c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 0x7f998c0c3fd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f998400c430 tx=0x7f998400c8f0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 shutdown_connections 2023-10-19T10:52:39.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f997806a050 0x7f997806c510 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f998c160240 0x7f998c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c15c660 0x7f998c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 --2- 172.21.15.50:0/3712918626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f998c0cd460 0x7f998c0c3fd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 >> 172.21.15.50:0/3712918626 conn(0x7f998c0cbc00 msgr2=0x7f998c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 shutdown_connections 2023-10-19T10:52:39.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.219+0000 7f9993900640 1 -- 172.21.15.50:0/3712918626 wait complete. 2023-10-19T10:52:39.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:52:39.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.340+0000 7fa06f4b2640 1 Processor -- start 2023-10-19T10:52:39.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.340+0000 7fa06f4b2640 1 -- start start 2023-10-19T10:52:39.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06f4b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680b4070 0x7fa0680b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06f4b2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa06815c960 0x7fa0680d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.348 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06f4b2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0680b4990 con 0x7fa06815cdc0 2023-10-19T10:52:39.348 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06f4b2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06815d280 con 0x7fa06815c960 2023-10-19T10:52:39.348 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06f4b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06815d560 con 0x7fa0680b4070 2023-10-19T10:52:39.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa06815c960 0x7fa0680d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa06815c960 0x7fa0680d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56000/0 (socket says 172.21.15.50:56000) 2023-10-19T10:52:39.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 -- 172.21.15.50:0/413511514 learned_addr learned my addr 172.21.15.50:0/413511514 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:39.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06d24e640 1 --2- 172.21.15.50:0/413511514 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680b4070 0x7fa0680b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 -- 172.21.15.50:0/413511514 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680b4070 msgr2=0x7fa0680b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:39.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 --2- 172.21.15.50:0/413511514 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680b4070 0x7fa0680b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 -- 172.21.15.50:0/413511514 >> v1:172.21.15.183:6789/0 conn(0x7fa06815cdc0 legacy=0x7fa068160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.341+0000 7fa06ca4d640 1 -- 172.21.15.50:0/413511514 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06815d840 con 0x7fa06815c960 2023-10-19T10:52:39.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.342+0000 7fa06ca4d640 1 --2- 172.21.15.50:0/413511514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa0680d2d70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009870 tx=0x7fa05c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ca65cf369cbbec server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.342+0000 7fa067fff640 1 -- 172.21.15.50:0/413511514 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c030840 con 0x7fa06815c960 2023-10-19T10:52:39.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.342+0000 7fa067fff640 1 -- 172.21.15.50:0/413511514 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c0309a0 con 0x7fa06815c960 2023-10-19T10:52:39.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.342+0000 7fa067fff640 1 -- 172.21.15.50:0/413511514 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c030d30 con 0x7fa06815c960 2023-10-19T10:52:39.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 -- 172.21.15.50:0/413511514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 msgr2=0x7fa0680d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.351 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/413511514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa0680d2d70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009870 tx=0x7fa05c02b980 comp rx=0 tx=0).stop 2023-10-19T10:52:39.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 -- 172.21.15.50:0/413511514 shutdown_connections 2023-10-19T10:52:39.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/413511514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa0680d2d70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/413511514 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680b4070 0x7fa0680b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 -- 172.21.15.50:0/413511514 >> 172.21.15.50:0/413511514 conn(0x7fa06809b970 msgr2=0x7fa0680d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 -- 172.21.15.50:0/413511514 shutdown_connections 2023-10-19T10:52:39.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.343+0000 7fa06f4b2640 1 -- 172.21.15.50:0/413511514 wait complete. 2023-10-19T10:52:39.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.344+0000 7fa06f4b2640 1 Processor -- start 2023-10-19T10:52:39.353 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.344+0000 7fa06f4b2640 1 -- start start 2023-10-19T10:52:39.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06f4b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0680b4070 0x7fa0681f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06f4b2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa06815edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.356 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06f4b2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa06815cdc0 0x7fa06815f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06f4b2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068145730 con 0x7fa06815c960 2023-10-19T10:52:39.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06f4b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681462b0 con 0x7fa0680b4070 2023-10-19T10:52:39.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06f4b2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa068145980 con 0x7fa06815cdc0 2023-10-19T10:52:39.357 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa06815edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa06815edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56004/0 (socket says 172.21.15.50:56004) 2023-10-19T10:52:39.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 -- 172.21.15.50:0/2076354900 learned_addr learned my addr 172.21.15.50:0/2076354900 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:39.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06da4f640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa06815cdc0 0x7fa06815f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.358 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06d24e640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0680b4070 0x7fa0681f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 -- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa06815cdc0 msgr2=0x7fa06815f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa06815cdc0 0x7fa06815f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 -- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0680b4070 msgr2=0x7fa0681f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0680b4070 0x7fa0681f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.359 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06ca4d640 1 -- 172.21.15.50:0/2076354900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0094d0 con 0x7fa06815c960 2023-10-19T10:52:39.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06da4f640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa06815cdc0 0x7fa06815f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:39.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.345+0000 7fa06d24e640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0680b4070 0x7fa0681f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:39.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.346+0000 7fa06ca4d640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa06815edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa05c02b9c0 tx=0x7fa05c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.346+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c039c90 con 0x7fa06815c960 2023-10-19T10:52:39.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.346+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c039df0 con 0x7fa06815c960 2023-10-19T10:52:39.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.346+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa068145c60 con 0x7fa06815c960 2023-10-19T10:52:39.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.346+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0446c0 con 0x7fa06815c960 2023-10-19T10:52:39.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.346+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0681f6370 con 0x7fa06815c960 2023-10-19T10:52:39.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.347+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06804e420 con 0x7fa06815c960 2023-10-19T10:52:39.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.348+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa05c04d3b0 con 0x7fa06815c960 2023-10-19T10:52:39.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.348+0000 7fa065ffb640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa05406a050 0x7fa05406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.348+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa05c03e070 con 0x7fa06815c960 2023-10-19T10:52:39.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.348+0000 7fa06d24e640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa05406a050 0x7fa05406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.349+0000 7fa06d24e640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa05406a050 0x7fa05406c510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa060009820 tx=0x7fa060008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.353+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa05c0dd940 con 0x7fa06815c960 2023-10-19T10:52:39.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.527+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0680b1720 con 0x7fa06815c960 2023-10-19T10:52:39.530 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.527+0000 7fa065ffb640 1 -- 172.21.15.50:0/2076354900 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa05c076750 con 0x7fa06815c960 2023-10-19T10:52:39.530 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:52:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa05406a050 msgr2=0x7fa05406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa05406a050 0x7fa05406c510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa060009820 tx=0x7fa060008040 comp rx=0 tx=0).stop 2023-10-19T10:52:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 msgr2=0x7fa06815edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa06815edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa05c02b9c0 tx=0x7fa05c004730 comp rx=0 tx=0).stop 2023-10-19T10:52:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 shutdown_connections 2023-10-19T10:52:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa05406a050 0x7fa05406c510 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa06815cdc0 0x7fa06815f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa06815c960 0x7fa06815edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 --2- 172.21.15.50:0/2076354900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0680b4070 0x7fa0681f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 >> 172.21.15.50:0/2076354900 conn(0x7fa06809b970 msgr2=0x7fa068160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 shutdown_connections 2023-10-19T10:52:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.529+0000 7fa06f4b2640 1 -- 172.21.15.50:0/2076354900 wait complete. 2023-10-19T10:52:39.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:52:39.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 Processor -- start 2023-10-19T10:52:39.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 -- start start 2023-10-19T10:52:39.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8ddc13d100 0x7f8ddc0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc13fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ddc13ffd0 0x7f8ddc06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc140800 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0cb4f0 con 0x7f8ddc13d100 2023-10-19T10:52:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.650+0000 7f8de2fa7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc15a730 con 0x7f8ddc0cb090 2023-10-19T10:52:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ddc13ffd0 0x7f8ddc06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ddc13ffd0 0x7f8ddc06b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56018/0 (socket says 172.21.15.50:56018) 2023-10-19T10:52:39.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 -- 172.21.15.50:0/1719341344 learned_addr learned my addr 172.21.15.50:0/1719341344 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de0d43640 1 --2- 172.21.15.50:0/1719341344 >> v2:172.21.15.183:3300/0 conn(0x7f8ddc13d100 0x7f8ddc0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 -- 172.21.15.50:0/1719341344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 msgr2=0x7f8ddc13fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 --2- 172.21.15.50:0/1719341344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc13fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 -- 172.21.15.50:0/1719341344 >> v2:172.21.15.183:3300/0 conn(0x7f8ddc13d100 msgr2=0x7f8ddc0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:39.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 --2- 172.21.15.50:0/1719341344 >> v2:172.21.15.183:3300/0 conn(0x7f8ddc13d100 0x7f8ddc0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de1544640 1 -- 172.21.15.50:0/1719341344 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc15b750 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.651+0000 7f8de0d43640 1 --2- 172.21.15.50:0/1719341344 >> v2:172.21.15.183:3300/0 conn(0x7f8ddc13d100 0x7f8ddc0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:39.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de1544640 1 --2- 172.21.15.50:0/1719341344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc06b5b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06da40 tx=0x7f8dd402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6465b47e10c455a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8ddb7fe640 1 -- 172.21.15.50:0/1719341344 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd402ca70 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8ddb7fe640 1 -- 172.21.15.50:0/1719341344 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd402cbd0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8ddb7fe640 1 -- 172.21.15.50:0/1719341344 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40343f0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 -- 172.21.15.50:0/1719341344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 msgr2=0x7f8ddc06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/1719341344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc06b5b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06da40 tx=0x7f8dd402bed0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 -- 172.21.15.50:0/1719341344 shutdown_connections 2023-10-19T10:52:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/1719341344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc06b5b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/1719341344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc13fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/1719341344 >> v2:172.21.15.183:3300/0 conn(0x7f8ddc13d100 0x7f8ddc0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 -- 172.21.15.50:0/1719341344 >> 172.21.15.50:0/1719341344 conn(0x7f8ddc06cfd0 msgr2=0x7f8ddc069800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.652+0000 7f8de2fa7640 1 -- 172.21.15.50:0/1719341344 shutdown_connections 2023-10-19T10:52:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.653+0000 7f8de2fa7640 1 -- 172.21.15.50:0/1719341344 wait complete. 2023-10-19T10:52:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 Processor -- start 2023-10-19T10:52:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 -- start start 2023-10-19T10:52:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc13d100 0x7f8ddc159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc13a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1f5010 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1f52f0 con 0x7f8ddc13d100 2023-10-19T10:52:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.654+0000 7f8de2fa7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1f55d0 con 0x7f8ddc0cb090 2023-10-19T10:52:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc13a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc13a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56034/0 (socket says 172.21.15.50:56034) 2023-10-19T10:52:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 -- 172.21.15.50:0/3897259105 learned_addr learned my addr 172.21.15.50:0/3897259105 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de0d43640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8ddbfff640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc13d100 0x7f8ddc159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 -- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 msgr2=0x7f8ddc0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc0adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 -- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc13d100 msgr2=0x7f8ddc159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc13d100 0x7f8ddc159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 -- 172.21.15.50:0/3897259105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4009b30 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de1544640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc13a690 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4036040 tx=0x7f8dd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4009510 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd403e450 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc1f58b0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd403e7e0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.655+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc0ae3e0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.656+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc1f5dd0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.669 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.657+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8dd403e940 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.657+0000 7f8dd97fa640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8dc006a050 0x7f8dc006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.657+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f8dd40ab4c0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.658+0000 7f8de0d43640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8dc006a050 0x7f8dc006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.658+0000 7f8de0d43640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8dc006a050 0x7f8dc006c510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000ab60 tx=0x7f8dd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.662+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f8ddc1f5dd0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.838 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.835+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ddc1f5dd0 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.839 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.835+0000 7f8dd97fa640 1 -- 172.21.15.50:0/3897259105 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8dd4043020 con 0x7f8ddc13ffd0 2023-10-19T10:52:39.839 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:52:39.840 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8dc006a050 msgr2=0x7f8dc006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8dc006a050 0x7f8dc006c510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000ab60 tx=0x7f8dd0008040 comp rx=0 tx=0).stop 2023-10-19T10:52:39.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 msgr2=0x7f8ddc13a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc13a690 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4036040 tx=0x7f8dd4004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:39.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 shutdown_connections 2023-10-19T10:52:39.841 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8dc006a050 0x7f8dc006c510 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ddc13ffd0 0x7f8ddc13a690 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc13d100 0x7f8ddc159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 --2- 172.21.15.50:0/3897259105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ddc0cb090 0x7f8ddc0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 >> 172.21.15.50:0/3897259105 conn(0x7f8ddc06cfd0 msgr2=0x7f8ddc13f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.842 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.837+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 shutdown_connections 2023-10-19T10:52:39.843 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.838+0000 7f8de2fa7640 1 -- 172.21.15.50:0/3897259105 wait complete. 2023-10-19T10:52:39.851 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:52:39.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.955+0000 7f8bc70d1640 1 Processor -- start 2023-10-19T10:52:39.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc70d1640 1 -- start start 2023-10-19T10:52:39.960 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc70d1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8bc015c570 0x7f8bc00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc70d1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c950 0x7f8bc00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc70d1640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc00cde60 con 0x7f8bc006b760 2023-10-19T10:52:39.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc70d1640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc006aa30 con 0x7f8bc015c570 2023-10-19T10:52:39.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc70d1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc006b2d0 con 0x7f8bc015c950 2023-10-19T10:52:39.961 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc4e6d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8bc015c570 0x7f8bc00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc566e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f8bc006b760 0x7f8bc00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50050/0 (socket says 172.21.15.50:50050) 2023-10-19T10:52:39.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.956+0000 7f8bc566e640 1 -- 172.21.15.50:0/3810344422 learned_addr learned my addr 172.21.15.50:0/3810344422 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:39.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.957+0000 7f8bbffff640 1 --2- 172.21.15.50:0/3810344422 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c950 0x7f8bc00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.957+0000 7f8bc4e6d640 1 -- 172.21.15.50:0/3810344422 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c950 msgr2=0x7f8bc00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:39.962 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.957+0000 7f8bc4e6d640 1 --2- 172.21.15.50:0/3810344422 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c950 0x7f8bc00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.957+0000 7f8bc4e6d640 1 -- 172.21.15.50:0/3810344422 >> v1:172.21.15.50:6789/0 conn(0x7f8bc006b760 legacy=0x7f8bc00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.957+0000 7f8bc4e6d640 1 -- 172.21.15.50:0/3810344422 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00ce090 con 0x7f8bc015c570 2023-10-19T10:52:39.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.957+0000 7f8bbf7fe640 1 -- 172.21.15.50:0/3810344422 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7469384 0 0) 0x7f8bc00cde60 con 0x7f8bc006b760 2023-10-19T10:52:39.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc4e6d640 1 --2- 172.21.15.50:0/3810344422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c570 0x7f8bc00d0e30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009de0 tx=0x7f8bb802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b92e32382ea919 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bbf7fe640 1 -- 172.21.15.50:0/3810344422 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb802cc00 con 0x7f8bc015c570 2023-10-19T10:52:39.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bbf7fe640 1 -- 172.21.15.50:0/3810344422 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb802cd60 con 0x7f8bc015c570 2023-10-19T10:52:39.964 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bbf7fe640 1 -- 172.21.15.50:0/3810344422 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8034660 con 0x7f8bc015c570 2023-10-19T10:52:39.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 -- 172.21.15.50:0/3810344422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c570 msgr2=0x7f8bc00d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.967 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/3810344422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c570 0x7f8bc00d0e30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009de0 tx=0x7f8bb802be50 comp rx=0 tx=0).stop 2023-10-19T10:52:39.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 -- 172.21.15.50:0/3810344422 shutdown_connections 2023-10-19T10:52:39.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/3810344422 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c950 0x7f8bc00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/3810344422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c570 0x7f8bc00d0e30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 -- 172.21.15.50:0/3810344422 >> 172.21.15.50:0/3810344422 conn(0x7f8bc0089ff0 msgr2=0x7f8bc009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:39.968 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 -- 172.21.15.50:0/3810344422 shutdown_connections 2023-10-19T10:52:39.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.958+0000 7f8bc70d1640 1 -- 172.21.15.50:0/3810344422 wait complete. 2023-10-19T10:52:39.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.959+0000 7f8bc70d1640 1 Processor -- start 2023-10-19T10:52:39.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 -- start start 2023-10-19T10:52:39.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc006b760 0x7f8bc01f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c570 0x7f8bc0144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 0x7f8bc0144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00b0c20 con 0x7f8bc015c950 2023-10-19T10:52:39.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00affa0 con 0x7f8bc006b760 2023-10-19T10:52:39.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc70d1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00b0280 con 0x7f8bc015c570 2023-10-19T10:52:39.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.960+0000 7f8bc566e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 0x7f8bc0144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 0x7f8bc0144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56042/0 (socket says 172.21.15.50:56042) 2023-10-19T10:52:39.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 -- 172.21.15.50:0/2846012455 learned_addr learned my addr 172.21.15.50:0/2846012455 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:39.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bbffff640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c570 0x7f8bc0144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc4e6d640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc006b760 0x7f8bc01f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 -- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c570 msgr2=0x7f8bc0144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c570 0x7f8bc0144560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 -- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc006b760 msgr2=0x7f8bc01f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc006b760 0x7f8bc01f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 -- 172.21.15.50:0/2846012455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8009a70 con 0x7f8bc015c950 2023-10-19T10:52:39.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc566e640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 0x7f8bc0144aa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000c410 tx=0x7f8bb000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00074e0 con 0x7f8bc015c950 2023-10-19T10:52:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb0007640 con 0x7f8bc015c950 2023-10-19T10:52:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc00b0560 con 0x7f8bc015c950 2023-10-19T10:52:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0004a20 con 0x7f8bc015c950 2023-10-19T10:52:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.961+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc01f4db0 con 0x7f8bc015c950 2023-10-19T10:52:39.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.962+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc006c190 con 0x7f8bc015c950 2023-10-19T10:52:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.963+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8bb0004b80 con 0x7f8bc015c950 2023-10-19T10:52:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.963+0000 7f8bbd7fa640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8bac06a050 0x7f8bac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.963+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f8bb00883f0 con 0x7f8bc015c950 2023-10-19T10:52:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.963+0000 7f8bc4e6d640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8bac06a050 0x7f8bac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:39.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.964+0000 7f8bc4e6d640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8bac06a050 0x7f8bac06c510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8036000 tx=0x7f8bb8030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:39.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:39.967+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f8bb00539c0 con 0x7f8bc015c950 2023-10-19T10:52:40.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.156+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8bc0073990 con 0x7f8bc015c950 2023-10-19T10:52:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.157+0000 7f8bbd7fa640 1 -- 172.21.15.50:0/2846012455 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v55) v1 ==== 80+0+56 (secure 0 0 0) 0x7f8bb0056fe0 con 0x7f8bc015c950 2023-10-19T10:52:40.160 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.1:0/216420809 2023-10-19T11:46:25.724828+0000 2023-10-19T10:52:40.160 INFO:teuthology.orchestra.run.smithi050.stderr:listed 1 entries 2023-10-19T10:52:40.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8bac06a050 msgr2=0x7f8bac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8bac06a050 0x7f8bac06c510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8036000 tx=0x7f8bb8030000 comp rx=0 tx=0).stop 2023-10-19T10:52:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 msgr2=0x7f8bc0144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 0x7f8bc0144aa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000c410 tx=0x7f8bb000c8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 shutdown_connections 2023-10-19T10:52:40.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f8bac06a050 0x7f8bac06c510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bc015c950 0x7f8bc0144aa0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bc015c570 0x7f8bc0144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 --2- 172.21.15.50:0/2846012455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc006b760 0x7f8bc01f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.158+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 >> 172.21.15.50:0/2846012455 conn(0x7f8bc0089ff0 msgr2=0x7f8bc0154230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:40.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.159+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 shutdown_connections 2023-10-19T10:52:40.164 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.159+0000 7f8bc70d1640 1 -- 172.21.15.50:0/2846012455 wait complete. 2023-10-19T10:52:40.172 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2023-10-19T10:52:40.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:40.173 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-19T10:52:40.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[109568]: fuse finished with error 0 and tester_r 0 2023-10-19T10:52:40.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:52:40.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:40.204 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-19T10:52:40.299 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:52:40.300 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:40.319 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:40.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.443+0000 7f5fa781c640 1 Processor -- start 2023-10-19T10:52:40.446 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.443+0000 7f5fa781c640 1 -- start start 2023-10-19T10:52:40.449 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa781c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa781c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5fa00b1860 0x7f5fa00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa781c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa0160670 con 0x7f5fa015c6e0 2023-10-19T10:52:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa781c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa0161280 con 0x7f5fa00b1860 2023-10-19T10:52:40.450 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa781c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa0160950 con 0x7f5fa00cda20 2023-10-19T10:52:40.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa5db9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5fa00b1860 0x7f5fa00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa4db7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa4db7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:57980/0 (socket says 172.21.15.50:57980) 2023-10-19T10:52:40.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa5db9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5fa00b1860 0x7f5fa00d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:58048/0 (socket says 172.21.15.50:58048) 2023-10-19T10:52:40.451 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa4db7640 1 -- 172.21.15.50:0/3348366918 learned_addr learned my addr 172.21.15.50:0/3348366918 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:40.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5f97fff640 1 -- 172.21.15.50:0/3348366918 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110353417 0 0) 0x7f5fa0160670 con 0x7f5fa015c6e0 2023-10-19T10:52:40.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5f97fff640 1 -- 172.21.15.50:0/3348366918 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f8c003520 con 0x7f5fa015c6e0 2023-10-19T10:52:40.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa5db9640 1 -- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 msgr2=0x7f5fa00d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:40.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa5db9640 1 --2- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa5db9640 1 -- 172.21.15.50:0/3348366918 >> v1:172.21.15.183:6789/0 conn(0x7f5fa015c6e0 legacy=0x7f5fa00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.452 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa5db9640 1 -- 172.21.15.50:0/3348366918 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa0160c30 con 0x7f5fa00b1860 2023-10-19T10:52:40.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.444+0000 7f5fa4db7640 1 --2- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:40.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.445+0000 7f5fa5db9640 1 --2- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa00b1860 0x7f5fa00d1200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009d90 tx=0x7f5f9802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=179d282f3efe461b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:40.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.445+0000 7f5f97fff640 1 -- 172.21.15.50:0/3348366918 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9802cb30 con 0x7f5fa00b1860 2023-10-19T10:52:40.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.445+0000 7f5f97fff640 1 -- 172.21.15.50:0/3348366918 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9802cc90 con 0x7f5fa00b1860 2023-10-19T10:52:40.453 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.445+0000 7f5f97fff640 1 -- 172.21.15.50:0/3348366918 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98034440 con 0x7f5fa00b1860 2023-10-19T10:52:40.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 -- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa00b1860 msgr2=0x7f5fa00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 --2- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa00b1860 0x7f5fa00d1200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009d90 tx=0x7f5f9802bd80 comp rx=0 tx=0).stop 2023-10-19T10:52:40.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 -- 172.21.15.50:0/3348366918 shutdown_connections 2023-10-19T10:52:40.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 --2- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa00b1860 0x7f5fa00d1200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.454 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 --2- 172.21.15.50:0/3348366918 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 -- 172.21.15.50:0/3348366918 >> 172.21.15.50:0/3348366918 conn(0x7f5fa006a530 msgr2=0x7f5fa006a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:40.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 -- 172.21.15.50:0/3348366918 shutdown_connections 2023-10-19T10:52:40.455 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.446+0000 7f5fa781c640 1 -- 172.21.15.50:0/3348366918 wait complete. 2023-10-19T10:52:40.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.447+0000 7f5fa781c640 1 Processor -- start 2023-10-19T10:52:40.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.447+0000 7f5fa781c640 1 -- start start 2023-10-19T10:52:40.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa781c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 0x7f5fa00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa781c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa781c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa015c6e0 0x7f5fa01508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.458 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa781c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00ca820 con 0x7f5fa00b1860 2023-10-19T10:52:40.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa781c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00b31b0 con 0x7f5fa015c6e0 2023-10-19T10:52:40.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa781c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00c9ba0 con 0x7f5fa00cda20 2023-10-19T10:52:40.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 0x7f5fa00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 0x7f5fa00c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56054/0 (socket says 172.21.15.50:56054) 2023-10-19T10:52:40.459 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 -- 172.21.15.50:0/744161463 learned_addr learned my addr 172.21.15.50:0/744161463 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:40.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa4db7640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa5db9640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa015c6e0 0x7f5fa01508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 -- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 msgr2=0x7f5fa00b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 -- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa015c6e0 msgr2=0x7f5fa01508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa015c6e0 0x7f5fa01508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 -- 172.21.15.50:0/744161463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900094d0 con 0x7f5fa00b1860 2023-10-19T10:52:40.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa4db7640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:40.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa5db9640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa015c6e0 0x7f5fa01508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:40.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.448+0000 7f5fa55b8640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 0x7f5fa00c94a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c500 tx=0x7f5f9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:40.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.449+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0052d0 con 0x7f5fa00b1860 2023-10-19T10:52:40.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.449+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c005450 con 0x7f5fa00b1860 2023-10-19T10:52:40.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.449+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f98009a70 con 0x7f5fa00b1860 2023-10-19T10:52:40.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.449+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c012870 con 0x7f5fa00b1860 2023-10-19T10:52:40.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.449+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa00ca160 con 0x7f5fa00b1860 2023-10-19T10:52:40.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.450+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa0073990 con 0x7f5fa00b1860 2023-10-19T10:52:40.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.450+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5f9c0129d0 con 0x7f5fa00b1860 2023-10-19T10:52:40.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.450+0000 7f5f95ffb640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8406a050 0x7f5f8406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.451+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f5f9c0883e0 con 0x7f5fa00b1860 2023-10-19T10:52:40.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.451+0000 7f5fa4db7640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8406a050 0x7f5f8406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.451+0000 7f5fa4db7640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8406a050 0x7f5f8406c510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5fa01f5400 tx=0x7f5f90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:40.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.455+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5f9c053930 con 0x7f5fa00b1860 2023-10-19T10:52:40.635 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.632+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fa00cc6e0 con 0x7f5fa00b1860 2023-10-19T10:52:40.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.633+0000 7f5f95ffb640 1 -- 172.21.15.50:0/744161463 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18701 (secure 0 0 0) 0x7f5f9c056f50 con 0x7f5fa00b1860 2023-10-19T10:52:40.636 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:40.637 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:46:26.319171+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8406a050 msgr2=0x7f5f8406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8406a050 0x7f5f8406c510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5fa01f5400 tx=0x7f5f90008040 comp rx=0 tx=0).stop 2023-10-19T10:52:40.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 msgr2=0x7f5fa00c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 0x7f5fa00c94a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c500 tx=0x7f5f9c00c9c0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 shutdown_connections 2023-10-19T10:52:40.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5f8406a050 0x7f5f8406c510 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa015c6e0 0x7f5fa01508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fa00cda20 0x7f5fa00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 --2- 172.21.15.50:0/744161463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fa00b1860 0x7f5fa00c94a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 >> 172.21.15.50:0/744161463 conn(0x7f5fa006a530 msgr2=0x7f5fa015f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 shutdown_connections 2023-10-19T10:52:40.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.635+0000 7f5fa781c640 1 -- 172.21.15.50:0/744161463 wait complete. 2023-10-19T10:52:40.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:38:26.398890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:38:26.910608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:52:40.650 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:52:40.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.757+0000 7fe2579bd640 1 Processor -- start 2023-10-19T10:52:40.760 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.757+0000 7fe2579bd640 1 -- start start 2023-10-19T10:52:40.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe2579bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2500b4070 0x7fe2500b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe2579bd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe25015c960 0x7fe2500d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe2579bd640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500b4990 con 0x7fe25015cdc0 2023-10-19T10:52:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe2579bd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25015d280 con 0x7fe25015c960 2023-10-19T10:52:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe2579bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25015d560 con 0x7fe2500b4070 2023-10-19T10:52:40.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe255f5a640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe25015cdc0 0x7fe250160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50052/0 (socket says 172.21.15.50:50052) 2023-10-19T10:52:40.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe255f5a640 1 -- 172.21.15.50:0/3610635834 learned_addr learned my addr 172.21.15.50:0/3610635834 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:40.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe255759640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2500b4070 0x7fe2500b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe254f58640 1 --2- 172.21.15.50:0/3610635834 >> v2:172.21.15.183:3300/0 conn(0x7fe25015c960 0x7fe2500d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe247fff640 1 -- 172.21.15.50:0/3610635834 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57161552 0 0) 0x7fe2500b4990 con 0x7fe25015cdc0 2023-10-19T10:52:40.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe247fff640 1 -- 172.21.15.50:0/3610635834 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe23c003520 con 0x7fe25015cdc0 2023-10-19T10:52:40.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe254f58640 1 -- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2500b4070 msgr2=0x7fe2500b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:40.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe254f58640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2500b4070 0x7fe2500b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe254f58640 1 -- 172.21.15.50:0/3610635834 >> v1:172.21.15.50:6789/0 conn(0x7fe25015cdc0 legacy=0x7fe250160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe254f58640 1 -- 172.21.15.50:0/3610635834 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25015d840 con 0x7fe25015c960 2023-10-19T10:52:40.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.758+0000 7fe255759640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2500b4070 0x7fe2500b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:40.768 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.759+0000 7fe254f58640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25015c960 0x7fe2500d2d70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe2400097a0 tx=0x7fe24002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7897e0fc96c617 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.759+0000 7fe247fff640 1 -- 172.21.15.50:0/3610635834 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240031830 con 0x7fe25015c960 2023-10-19T10:52:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.759+0000 7fe247fff640 1 -- 172.21.15.50:0/3610635834 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe240031990 con 0x7fe25015c960 2023-10-19T10:52:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.759+0000 7fe247fff640 1 -- 172.21.15.50:0/3610635834 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240031d00 con 0x7fe25015c960 2023-10-19T10:52:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 -- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25015c960 msgr2=0x7fe2500d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25015c960 0x7fe2500d2d70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe2400097a0 tx=0x7fe24002b8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 -- 172.21.15.50:0/3610635834 shutdown_connections 2023-10-19T10:52:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25015c960 0x7fe2500d2d70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 --2- 172.21.15.50:0/3610635834 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2500b4070 0x7fe2500b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 -- 172.21.15.50:0/3610635834 >> 172.21.15.50:0/3610635834 conn(0x7fe25009b970 msgr2=0x7fe2500d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 -- 172.21.15.50:0/3610635834 shutdown_connections 2023-10-19T10:52:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.760+0000 7fe2579bd640 1 -- 172.21.15.50:0/3610635834 wait complete. 2023-10-19T10:52:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.761+0000 7fe2579bd640 1 Processor -- start 2023-10-19T10:52:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.761+0000 7fe2579bd640 1 -- start start 2023-10-19T10:52:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe2579bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2500b4070 0x7fe2501f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe2579bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 0x7fe2500b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe2579bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe25015cdc0 0x7fe2500b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe2579bd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2501436f0 con 0x7fe25015c960 2023-10-19T10:52:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe2579bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2501f1b10 con 0x7fe2500b4070 2023-10-19T10:52:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe2579bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2500c9f60 con 0x7fe25015cdc0 2023-10-19T10:52:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 0x7fe2500b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 0x7fe2500b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56068/0 (socket says 172.21.15.50:56068) 2023-10-19T10:52:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 -- 172.21.15.50:0/1312624991 learned_addr learned my addr 172.21.15.50:0/1312624991 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe255f5a640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe25015cdc0 0x7fe2500b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe255759640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2500b4070 0x7fe2501f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 -- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe25015cdc0 msgr2=0x7fe2500b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe25015cdc0 0x7fe2500b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 -- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2500b4070 msgr2=0x7fe2501f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2500b4070 0x7fe2501f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe254f58640 1 -- 172.21.15.50:0/1312624991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0093a0 con 0x7fe25015c960 2023-10-19T10:52:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe255759640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2500b4070 0x7fe2501f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.762+0000 7fe255f5a640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe25015cdc0 0x7fe2500b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.763+0000 7fe254f58640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 0x7fe2500b5210 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe2400098c0 tx=0x7fe2400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.763+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24003a630 con 0x7fe25015c960 2023-10-19T10:52:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.763+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24003a790 con 0x7fe25015c960 2023-10-19T10:52:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.763+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe240009400 con 0x7fe25015c960 2023-10-19T10:52:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.763+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24003ab40 con 0x7fe25015c960 2023-10-19T10:52:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.763+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2500ca7e0 con 0x7fe25015c960 2023-10-19T10:52:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.764+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25004e420 con 0x7fe25015c960 2023-10-19T10:52:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.764+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe24003aca0 con 0x7fe25015c960 2023-10-19T10:52:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.765+0000 7fe245ffb640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe23406a050 0x7fe23406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.765+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fe2400aa360 con 0x7fe25015c960 2023-10-19T10:52:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.765+0000 7fe255759640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe23406a050 0x7fe23406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.765+0000 7fe255759640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe23406a050 0x7fe23406c510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2500b82a0 tx=0x7fe24c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.769+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe2400758a0 con 0x7fe25015c960 2023-10-19T10:52:40.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.943+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2500b4ce0 con 0x7fe25015c960 2023-10-19T10:52:40.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.944+0000 7fe245ffb640 1 -- 172.21.15.50:0/1312624991 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe240078ec0 con 0x7fe25015c960 2023-10-19T10:52:40.947 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:52:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.945+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe23406a050 msgr2=0x7fe23406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.945+0000 7fe2579bd640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe23406a050 0x7fe23406c510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2500b82a0 tx=0x7fe24c008040 comp rx=0 tx=0).stop 2023-10-19T10:52:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.945+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 msgr2=0x7fe2500b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:40.949 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.945+0000 7fe2579bd640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 0x7fe2500b5210 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe2400098c0 tx=0x7fe2400046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 shutdown_connections 2023-10-19T10:52:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe23406a050 0x7fe23406c510 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe25015cdc0 0x7fe2500b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe25015c960 0x7fe2500b5210 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 --2- 172.21.15.50:0/1312624991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2500b4070 0x7fe2501f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:40.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 >> 172.21.15.50:0/1312624991 conn(0x7fe25009b970 msgr2=0x7fe25006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:40.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 shutdown_connections 2023-10-19T10:52:40.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:40.946+0000 7fe2579bd640 1 -- 172.21.15.50:0/1312624991 wait complete. 2023-10-19T10:52:40.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:52:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.065+0000 7fc6c05f0640 1 Processor -- start 2023-10-19T10:52:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.066+0000 7fc6c05f0640 1 -- start start 2023-10-19T10:52:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.066+0000 7fc6c05f0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b38f0 0x7fc6b80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.066+0000 7fc6c05f0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80af250 con 0x7fc6b80b3090 2023-10-19T10:52:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.066+0000 7fc6c05f0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80af510 con 0x7fc6b8161210 2023-10-19T10:52:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.066+0000 7fc6c05f0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80b0190 con 0x7fc6b80b38f0 2023-10-19T10:52:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6bdb8b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc6b80b3090 0x7fc6b8160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50066/0 (socket says 172.21.15.50:50066) 2023-10-19T10:52:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6bdb8b640 1 -- 172.21.15.50:0/1030060462 learned_addr learned my addr 172.21.15.50:0/1030060462 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6be38c640 1 --2- 172.21.15.50:0/1030060462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b38f0 0x7fc6b80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6bd38a640 1 -- 172.21.15.50:0/1030060462 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005340933 0 0) 0x7fc6b80af250 con 0x7fc6b80b3090 2023-10-19T10:52:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6bd38a640 1 -- 172.21.15.50:0/1030060462 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a0003520 con 0x7fc6b80b3090 2023-10-19T10:52:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6bd38a640 1 -- 172.21.15.50:0/1030060462 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406254363 0 0) 0x7fc6b80af510 con 0x7fc6b8161210 2023-10-19T10:52:41.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6be38c640 1 -- 172.21.15.50:0/1030060462 >> v1:172.21.15.183:6789/0 conn(0x7fc6b8161210 legacy=0x7fc6b80b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6be38c640 1 -- 172.21.15.50:0/1030060462 >> v1:172.21.15.50:6789/0 conn(0x7fc6b80b3090 legacy=0x7fc6b8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.067+0000 7fc6be38c640 1 -- 172.21.15.50:0/1030060462 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80af7f0 con 0x7fc6b80b38f0 2023-10-19T10:52:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.068+0000 7fc6be38c640 1 --2- 172.21.15.50:0/1030060462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b38f0 0x7fc6b80d30e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097c0 tx=0x7fc6b002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e3ed6d30619cb52 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.068+0000 7fc6bd38a640 1 -- 172.21.15.50:0/1030060462 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0031890 con 0x7fc6b80b38f0 2023-10-19T10:52:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.068+0000 7fc6bd38a640 1 -- 172.21.15.50:0/1030060462 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b00319f0 con 0x7fc6b80b38f0 2023-10-19T10:52:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.068+0000 7fc6bd38a640 1 -- 172.21.15.50:0/1030060462 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0031d80 con 0x7fc6b80b38f0 2023-10-19T10:52:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1030060462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b38f0 msgr2=0x7fc6b80d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1030060462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b38f0 0x7fc6b80d30e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097c0 tx=0x7fc6b002b930 comp rx=0 tx=0).stop 2023-10-19T10:52:41.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1030060462 shutdown_connections 2023-10-19T10:52:41.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1030060462 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b38f0 0x7fc6b80d30e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1030060462 >> 172.21.15.50:0/1030060462 conn(0x7fc6b809b960 msgr2=0x7fc6b80d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:41.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1030060462 shutdown_connections 2023-10-19T10:52:41.077 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.069+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1030060462 wait complete. 2023-10-19T10:52:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.070+0000 7fc6c05f0640 1 Processor -- start 2023-10-19T10:52:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.070+0000 7fc6c05f0640 1 -- start start 2023-10-19T10:52:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6c05f0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b3090 0x7fc6b80ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6c05f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 0x7fc6b80b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6c05f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b8161210 0x7fc6b80b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6c05f0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b815df40 con 0x7fc6b80b38f0 2023-10-19T10:52:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6c05f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b815e220 con 0x7fc6b8161210 2023-10-19T10:52:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6c05f0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b815e500 con 0x7fc6b80b3090 2023-10-19T10:52:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 0x7fc6b80b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 0x7fc6b80b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56080/0 (socket says 172.21.15.50:56080) 2023-10-19T10:52:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 -- 172.21.15.50:0/1104385261 learned_addr learned my addr 172.21.15.50:0/1104385261 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6be38c640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b3090 0x7fc6b80ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 -- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b3090 msgr2=0x7fc6b80ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b3090 0x7fc6b80ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 -- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b8161210 msgr2=0x7fc6b80b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b8161210 0x7fc6b80b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.071+0000 7fc6bdb8b640 1 -- 172.21.15.50:0/1104385261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00093a0 con 0x7fc6b80b38f0 2023-10-19T10:52:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.072+0000 7fc6bdb8b640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 0x7fc6b80b1c90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c400 tx=0x7fc6ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.072+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac005ce0 con 0x7fc6b80b38f0 2023-10-19T10:52:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.072+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac005e60 con 0x7fc6b80b38f0 2023-10-19T10:52:41.081 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.072+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b815e7e0 con 0x7fc6b80b38f0 2023-10-19T10:52:41.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.072+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0072f0 con 0x7fc6b80b38f0 2023-10-19T10:52:41.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.072+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b815c090 con 0x7fc6b80b38f0 2023-10-19T10:52:41.082 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.073+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b815ed00 con 0x7fc6b80b38f0 2023-10-19T10:52:41.083 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.073+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc6ac007450 con 0x7fc6b80b38f0 2023-10-19T10:52:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.074+0000 7fc6b6ffd640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc6a406a050 0x7fc6a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.074+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fc6ac087e70 con 0x7fc6b80b38f0 2023-10-19T10:52:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.074+0000 7fc6be38c640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc6a406a050 0x7fc6a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.074+0000 7fc6be38c640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc6a406a050 0x7fc6a406c510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0036000 tx=0x7fc6b0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.084 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.078+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc6b815ed00 con 0x7fc6b80b38f0 2023-10-19T10:52:41.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.252+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6b815ed00 con 0x7fc6b80b38f0 2023-10-19T10:52:41.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.252+0000 7fc6b6ffd640 1 -- 172.21.15.50:0/1104385261 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6ac017020 con 0x7fc6b80b38f0 2023-10-19T10:52:41.256 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:52:41.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc6a406a050 msgr2=0x7fc6a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc6a406a050 0x7fc6a406c510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0036000 tx=0x7fc6b0008030 comp rx=0 tx=0).stop 2023-10-19T10:52:41.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 msgr2=0x7fc6b80b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 0x7fc6b80b1c90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c400 tx=0x7fc6ac00c8c0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 shutdown_connections 2023-10-19T10:52:41.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc6a406a050 0x7fc6a406c510 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b8161210 0x7fc6b80b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b80b38f0 0x7fc6b80b1c90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 --2- 172.21.15.50:0/1104385261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6b80b3090 0x7fc6b80ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 >> 172.21.15.50:0/1104385261 conn(0x7fc6b809b960 msgr2=0x7fc6b8072180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 shutdown_connections 2023-10-19T10:52:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.254+0000 7fc6c05f0640 1 -- 172.21.15.50:0/1104385261 wait complete. 2023-10-19T10:52:41.268 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:52:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.376+0000 7f5c12834640 1 Processor -- start 2023-10-19T10:52:41.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.376+0000 7f5c12834640 1 -- start start 2023-10-19T10:52:41.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c12834640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5c0c15c960 0x7f5c0c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c12834640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5c0c15cdf0 0x7f5c0c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c12834640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c15d280 con 0x7f5c0c0b4070 2023-10-19T10:52:41.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c12834640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c15d560 con 0x7f5c0c15cdf0 2023-10-19T10:52:41.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c12834640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c15d840 con 0x7f5c0c15c960 2023-10-19T10:52:41.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5c0c15cdf0 0x7f5c0c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5c0c15cdf0 0x7f5c0c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56092/0 (socket says 172.21.15.50:56092) 2023-10-19T10:52:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 -- 172.21.15.50:0/46434225 learned_addr learned my addr 172.21.15.50:0/46434225 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c0b7fe640 1 --2- 172.21.15.50:0/46434225 >> v2:172.21.15.183:3300/0 conn(0x7f5c0c15c960 0x7f5c0c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 -- 172.21.15.50:0/46434225 >> v2:172.21.15.183:3300/0 conn(0x7f5c0c15c960 msgr2=0x7f5c0c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:41.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 --2- 172.21.15.50:0/46434225 >> v2:172.21.15.183:3300/0 conn(0x7f5c0c15c960 0x7f5c0c0d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 -- 172.21.15.50:0/46434225 >> v1:172.21.15.183:6789/0 conn(0x7f5c0c0b4070 legacy=0x7f5c0c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.383 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.377+0000 7f5c10dd1640 1 -- 172.21.15.50:0/46434225 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c15db20 con 0x7f5c0c15cdf0 2023-10-19T10:52:41.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.378+0000 7f5c10dd1640 1 --2- 172.21.15.50:0/46434225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15cdf0 0x7f5c0c0cd4b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0d05a0 tx=0x7f5c0402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38e06d0548e1343 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.378+0000 7f5c0affd640 1 -- 172.21.15.50:0/46434225 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0402cb20 con 0x7f5c0c15cdf0 2023-10-19T10:52:41.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.378+0000 7f5c0affd640 1 -- 172.21.15.50:0/46434225 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0402cc80 con 0x7f5c0c15cdf0 2023-10-19T10:52:41.387 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.378+0000 7f5c0affd640 1 -- 172.21.15.50:0/46434225 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040343f0 con 0x7f5c0c15cdf0 2023-10-19T10:52:41.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 -- 172.21.15.50:0/46434225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15cdf0 msgr2=0x7f5c0c0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 --2- 172.21.15.50:0/46434225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15cdf0 0x7f5c0c0cd4b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0d05a0 tx=0x7f5c0402bf80 comp rx=0 tx=0).stop 2023-10-19T10:52:41.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 -- 172.21.15.50:0/46434225 shutdown_connections 2023-10-19T10:52:41.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 --2- 172.21.15.50:0/46434225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15cdf0 0x7f5c0c0cd4b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.388 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 --2- 172.21.15.50:0/46434225 >> v2:172.21.15.183:3300/0 conn(0x7f5c0c15c960 0x7f5c0c0d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 -- 172.21.15.50:0/46434225 >> 172.21.15.50:0/46434225 conn(0x7f5c0c09b970 msgr2=0x7f5c0c0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 -- 172.21.15.50:0/46434225 shutdown_connections 2023-10-19T10:52:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.379+0000 7f5c12834640 1 -- 172.21.15.50:0/46434225 wait complete. 2023-10-19T10:52:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.380+0000 7f5c12834640 1 Processor -- start 2023-10-19T10:52:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.380+0000 7f5c12834640 1 -- start start 2023-10-19T10:52:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c12834640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c0c0b4070 0x7f5c0c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c12834640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 0x7f5c0c0b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c12834640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c0c0b31f0 0x7f5c0c1f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c12834640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1f5560 con 0x7f5c0c15c960 2023-10-19T10:52:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c12834640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0c65a0 con 0x7f5c0c0b4070 2023-10-19T10:52:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c12834640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0c8e20 con 0x7f5c0c0b31f0 2023-10-19T10:52:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 0x7f5c0c0b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 0x7f5c0c0b64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56096/0 (socket says 172.21.15.50:56096) 2023-10-19T10:52:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 -- 172.21.15.50:0/1544325247 learned_addr learned my addr 172.21.15.50:0/1544325247 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0bfff640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c0c0b4070 0x7f5c0c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c10dd1640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c0c0b31f0 0x7f5c0c1f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 -- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c0c0b31f0 msgr2=0x7f5c0c1f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c0c0b31f0 0x7f5c0c1f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 -- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c0c0b4070 msgr2=0x7f5c0c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c0c0b4070 0x7f5c0c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 -- 172.21.15.50:0/1544325247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009b30 con 0x7f5c0c15c960 2023-10-19T10:52:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0bfff640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c0c0b4070 0x7f5c0c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.381+0000 7f5c0b7fe640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 0x7f5c0c0b64d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00d800 tx=0x7f5bfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.382+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc00b6c0 con 0x7f5c0c15c960 2023-10-19T10:52:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.382+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc00b820 con 0x7f5c0c15c960 2023-10-19T10:52:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.382+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c0c9100 con 0x7f5c0c15c960 2023-10-19T10:52:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.382+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc00bbb0 con 0x7f5c0c15c960 2023-10-19T10:52:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.382+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c0b6a10 con 0x7f5c0c15c960 2023-10-19T10:52:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.383+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c1f52c0 con 0x7f5c0c15c960 2023-10-19T10:52:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.383+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5bfc00bd10 con 0x7f5c0c15c960 2023-10-19T10:52:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.383+0000 7f5c08ff9640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5bec06a050 0x7f5bec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.394 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.383+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f5bfc088c00 con 0x7f5c0c15c960 2023-10-19T10:52:41.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.384+0000 7f5c0bfff640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5bec06a050 0x7f5bec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.384+0000 7f5c0bfff640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5bec06a050 0x7f5bec06c510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4000fb0 tx=0x7f5bf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.395 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.388+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5c0c1f52c0 con 0x7f5c0c15c960 2023-10-19T10:52:41.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.564+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c0c1f52c0 con 0x7f5c0c15c960 2023-10-19T10:52:41.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.564+0000 7f5c08ff9640 1 -- 172.21.15.50:0/1544325247 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c0c1f52c0 con 0x7f5c0c15c960 2023-10-19T10:52:41.568 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:52:41.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5bec06a050 msgr2=0x7f5bec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5bec06a050 0x7f5bec06c510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4000fb0 tx=0x7f5bf4009040 comp rx=0 tx=0).stop 2023-10-19T10:52:41.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 msgr2=0x7f5c0c0b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 0x7f5c0c0b64d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00d800 tx=0x7f5bfc00dcc0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 shutdown_connections 2023-10-19T10:52:41.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5bec06a050 0x7f5bec06c510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c0c0b31f0 0x7f5c0c1f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c0c15c960 0x7f5c0c0b64d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 --2- 172.21.15.50:0/1544325247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c0c0b4070 0x7f5c0c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 >> 172.21.15.50:0/1544325247 conn(0x7f5c0c09b970 msgr2=0x7f5c0c06a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:41.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 shutdown_connections 2023-10-19T10:52:41.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.566+0000 7f5c12834640 1 -- 172.21.15.50:0/1544325247 wait complete. 2023-10-19T10:52:41.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:52:41.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.686+0000 7f80e01bd640 1 Processor -- start 2023-10-19T10:52:41.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.686+0000 7f80e01bd640 1 -- start start 2023-10-19T10:52:41.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80e01bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b38f0 0x7f80d80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80e01bd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f80d80b3090 0x7f80d8160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80e01bd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f80d80b3cd0 0x7f80d8161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80e01bd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d80b0190 con 0x7f80d80b3cd0 2023-10-19T10:52:41.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80e01bd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d80af250 con 0x7f80d80b3090 2023-10-19T10:52:41.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80e01bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80af510 con 0x7f80d80b38f0 2023-10-19T10:52:41.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80de75a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f80d80b3cd0 0x7f80d8161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80de75a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f80d80b3cd0 0x7f80d8161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56108/0 (socket says 172.21.15.50:56108) 2023-10-19T10:52:41.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.687+0000 7f80de75a640 1 -- 172.21.15.50:0/3533370886 learned_addr learned my addr 172.21.15.50:0/3533370886 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:41.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80de75a640 1 -- 172.21.15.50:0/3533370886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b38f0 msgr2=0x7f80d80d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:41.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80ddf59640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b38f0 0x7f80d80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80dd758640 1 --2- 172.21.15.50:0/3533370886 >> v2:172.21.15.183:3300/0 conn(0x7f80d80b3090 0x7f80d8160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80de75a640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b38f0 0x7f80d80d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80de75a640 1 -- 172.21.15.50:0/3533370886 >> v2:172.21.15.183:3300/0 conn(0x7f80d80b3090 msgr2=0x7f80d8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80de75a640 1 --2- 172.21.15.50:0/3533370886 >> v2:172.21.15.183:3300/0 conn(0x7f80d80b3090 0x7f80d8160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80de75a640 1 -- 172.21.15.50:0/3533370886 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80af7f0 con 0x7f80d80b3cd0 2023-10-19T10:52:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80ddf59640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b38f0 0x7f80d80d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80dd758640 1 --2- 172.21.15.50:0/3533370886 >> v2:172.21.15.183:3300/0 conn(0x7f80d80b3090 0x7f80d8160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.688+0000 7f80de75a640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b3cd0 0x7f80d8161040 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f80d808b140 tx=0x7f80d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b989f3b139ade41 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80dcf57640 1 -- 172.21.15.50:0/3533370886 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0031870 con 0x7f80d80b3cd0 2023-10-19T10:52:41.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80dcf57640 1 -- 172.21.15.50:0/3533370886 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d00319d0 con 0x7f80d80b3cd0 2023-10-19T10:52:41.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80dcf57640 1 -- 172.21.15.50:0/3533370886 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0031d60 con 0x7f80d80b3cd0 2023-10-19T10:52:41.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 -- 172.21.15.50:0/3533370886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b3cd0 msgr2=0x7f80d8161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b3cd0 0x7f80d8161040 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f80d808b140 tx=0x7f80d002b910 comp rx=0 tx=0).stop 2023-10-19T10:52:41.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 -- 172.21.15.50:0/3533370886 shutdown_connections 2023-10-19T10:52:41.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b3cd0 0x7f80d8161040 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 --2- 172.21.15.50:0/3533370886 >> v2:172.21.15.183:3300/0 conn(0x7f80d80b3090 0x7f80d8160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 --2- 172.21.15.50:0/3533370886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b38f0 0x7f80d80d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 -- 172.21.15.50:0/3533370886 >> 172.21.15.50:0/3533370886 conn(0x7f80d809b960 msgr2=0x7f80d80d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:41.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 -- 172.21.15.50:0/3533370886 shutdown_connections 2023-10-19T10:52:41.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.689+0000 7f80e01bd640 1 -- 172.21.15.50:0/3533370886 wait complete. 2023-10-19T10:52:41.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.690+0000 7f80e01bd640 1 Processor -- start 2023-10-19T10:52:41.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.690+0000 7f80e01bd640 1 -- start start 2023-10-19T10:52:41.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80e01bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80d80b3090 0x7f80d8148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80e01bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 0x7f80d8158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80e01bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b3cd0 0x7f80d815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80e01bd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d815a990 con 0x7f80d80b38f0 2023-10-19T10:52:41.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80e01bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d815b7b0 con 0x7f80d80b3090 2023-10-19T10:52:41.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80e01bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d815aba0 con 0x7f80d80b3cd0 2023-10-19T10:52:41.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 0x7f80d8158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 0x7f80d8158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56112/0 (socket says 172.21.15.50:56112) 2023-10-19T10:52:41.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 -- 172.21.15.50:0/990726041 learned_addr learned my addr 172.21.15.50:0/990726041 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:41.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80ddf59640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80d80b3090 0x7f80d8148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80de75a640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b3cd0 0x7f80d815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 -- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b3cd0 msgr2=0x7f80d815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b3cd0 0x7f80d815bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 -- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80d80b3090 msgr2=0x7f80d8148c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80d80b3090 0x7f80d8148c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.691+0000 7f80dd758640 1 -- 172.21.15.50:0/990726041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d00094d0 con 0x7f80d80b38f0 2023-10-19T10:52:41.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80ddf59640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80d80b3090 0x7f80d8148c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:41.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80de75a640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b3cd0 0x7f80d815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:41.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80dd758640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 0x7f80d8158ce0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c7c0 tx=0x7f80cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007820 con 0x7f80d80b38f0 2023-10-19T10:52:41.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc007980 con 0x7f80d80b38f0 2023-10-19T10:52:41.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d815ae80 con 0x7f80d80b38f0 2023-10-19T10:52:41.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007d10 con 0x7f80d80b38f0 2023-10-19T10:52:41.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.692+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d815b440 con 0x7f80d80b38f0 2023-10-19T10:52:41.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.693+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d815b920 con 0x7f80d80b38f0 2023-10-19T10:52:41.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.693+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f80cc0113f0 con 0x7f80d80b38f0 2023-10-19T10:52:41.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.694+0000 7f80d67fc640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f80c406a050 0x7f80c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.694+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f80cc088900 con 0x7f80d80b38f0 2023-10-19T10:52:41.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.694+0000 7f80ddf59640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f80c406a050 0x7f80c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:41.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.694+0000 7f80ddf59640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f80c406a050 0x7f80c406c510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b450 tx=0x7f80c8007580 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:41.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.698+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f80cc053da0 con 0x7f80d80b38f0 2023-10-19T10:52:41.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.873+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80d804e420 con 0x7f80d80b38f0 2023-10-19T10:52:41.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.874+0000 7f80d67fc640 1 -- 172.21.15.50:0/990726041 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80cc0573c0 con 0x7f80d80b38f0 2023-10-19T10:52:41.877 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:52:41.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f80c406a050 msgr2=0x7f80c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f80c406a050 0x7f80c406c510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b450 tx=0x7f80c8007580 comp rx=0 tx=0).stop 2023-10-19T10:52:41.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 msgr2=0x7f80d8158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:41.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 0x7f80d8158ce0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c7c0 tx=0x7f80cc00cc80 comp rx=0 tx=0).stop 2023-10-19T10:52:41.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 shutdown_connections 2023-10-19T10:52:41.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f80c406a050 0x7f80c406c510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f80d80b3cd0 0x7f80d815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80d80b38f0 0x7f80d8158ce0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 --2- 172.21.15.50:0/990726041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80d80b3090 0x7f80d8148c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:41.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 >> 172.21.15.50:0/990726041 conn(0x7f80d809b960 msgr2=0x7f80d80d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:41.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 shutdown_connections 2023-10-19T10:52:41.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.876+0000 7f80e01bd640 1 -- 172.21.15.50:0/990726041 wait complete. 2023-10-19T10:52:41.890 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-19T10:52:41.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.994+0000 7ff8685eb640 1 Processor -- start 2023-10-19T10:52:41.997 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.994+0000 7ff8685eb640 1 -- start start 2023-10-19T10:52:41.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff8685eb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff86015c570 0x7ff8600d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff8685eb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff8600cd920 0x7ff8600d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:41.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff8685eb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86006aa30 con 0x7ff86015c950 2023-10-19T10:52:41.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff8685eb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86006b2d0 con 0x7ff8600cd920 2023-10-19T10:52:41.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff8685eb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8600ce090 con 0x7ff86015c570 2023-10-19T10:52:42.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff866b88640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff8600cd920 0x7ff8600d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff865b86640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff86015c950 0x7ff86006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50082/0 (socket says 172.21.15.50:50082) 2023-10-19T10:52:42.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff865b86640 1 -- 172.21.15.50:0/705673696 learned_addr learned my addr 172.21.15.50:0/705673696 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:42.000 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff866387640 1 --2- 172.21.15.50:0/705673696 >> v2:172.21.15.183:3300/0 conn(0x7ff86015c570 0x7ff8600d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff866b88640 1 -- 172.21.15.50:0/705673696 >> v2:172.21.15.183:3300/0 conn(0x7ff86015c570 msgr2=0x7ff8600d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:42.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff866b88640 1 --2- 172.21.15.50:0/705673696 >> v2:172.21.15.183:3300/0 conn(0x7ff86015c570 0x7ff8600d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff866b88640 1 -- 172.21.15.50:0/705673696 >> v1:172.21.15.50:6789/0 conn(0x7ff86015c950 legacy=0x7ff86006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff866b88640 1 -- 172.21.15.50:0/705673696 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86006ac80 con 0x7ff8600cd920 2023-10-19T10:52:42.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.995+0000 7ff865385640 1 -- 172.21.15.50:0/705673696 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787711223 0 0) 0x7ff86006aa30 con 0x7ff86015c950 2023-10-19T10:52:42.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.996+0000 7ff866b88640 1 --2- 172.21.15.50:0/705673696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff8600d26a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff8600d2670 tx=0x7ff85802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c073a454ce8813d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:42.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.996+0000 7ff865385640 1 -- 172.21.15.50:0/705673696 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85802cbf0 con 0x7ff8600cd920 2023-10-19T10:52:42.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.996+0000 7ff865385640 1 -- 172.21.15.50:0/705673696 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85802cd50 con 0x7ff8600cd920 2023-10-19T10:52:42.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.996+0000 7ff865385640 1 -- 172.21.15.50:0/705673696 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858034660 con 0x7ff8600cd920 2023-10-19T10:52:42.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 -- 172.21.15.50:0/705673696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 msgr2=0x7ff8600d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 --2- 172.21.15.50:0/705673696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff8600d26a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff8600d2670 tx=0x7ff85802be40 comp rx=0 tx=0).stop 2023-10-19T10:52:42.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 -- 172.21.15.50:0/705673696 shutdown_connections 2023-10-19T10:52:42.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 --2- 172.21.15.50:0/705673696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff8600d26a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 --2- 172.21.15.50:0/705673696 >> v2:172.21.15.183:3300/0 conn(0x7ff86015c570 0x7ff8600d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 -- 172.21.15.50:0/705673696 >> 172.21.15.50:0/705673696 conn(0x7ff860089ff0 msgr2=0x7ff86009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:42.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 -- 172.21.15.50:0/705673696 shutdown_connections 2023-10-19T10:52:42.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.997+0000 7ff8685eb640 1 -- 172.21.15.50:0/705673696 wait complete. 2023-10-19T10:52:42.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.998+0000 7ff8685eb640 1 Processor -- start 2023-10-19T10:52:42.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.998+0000 7ff8685eb640 1 -- start start 2023-10-19T10:52:42.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff8685eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff86015b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff8685eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86015c570 0x7ff86009c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff8685eb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff86015c950 0x7ff86015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff8685eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8600c7c60 con 0x7ff8600cd920 2023-10-19T10:52:42.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff8685eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86015ba60 con 0x7ff86015c570 2023-10-19T10:52:42.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff8685eb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8600c7330 con 0x7ff86015c950 2023-10-19T10:52:42.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff86015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff86015b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56134/0 (socket says 172.21.15.50:56134) 2023-10-19T10:52:42.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 -- 172.21.15.50:0/207361236 learned_addr learned my addr 172.21.15.50:0/207361236 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:42.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff865b86640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86015c570 0x7ff86009c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866b88640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff86015c950 0x7ff86015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 -- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff86015c950 msgr2=0x7ff86015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff86015c950 0x7ff86015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 -- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86015c570 msgr2=0x7ff86009c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86015c570 0x7ff86009c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 -- 172.21.15.50:0/207361236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858009a70 con 0x7ff8600cd920 2023-10-19T10:52:42.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff865b86640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86015c570 0x7ff86009c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:42.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff866387640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff86015b520 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00d800 tx=0x7ff85c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:42.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0096c0 con 0x7ff8600cd920 2023-10-19T10:52:42.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c009820 con 0x7ff8600cd920 2023-10-19T10:52:42.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:41.999+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c009bd0 con 0x7ff8600cd920 2023-10-19T10:52:42.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.000+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8600c7610 con 0x7ff8600cd920 2023-10-19T10:52:42.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.000+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8600cc610 con 0x7ff8600cd920 2023-10-19T10:52:42.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.000+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff860073990 con 0x7ff8600cd920 2023-10-19T10:52:42.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.001+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff85c009d30 con 0x7ff8600cd920 2023-10-19T10:52:42.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.001+0000 7ff856ffd640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff84406a080 0x7ff84406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.001+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff85c088ef0 con 0x7ff8600cd920 2023-10-19T10:52:42.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.002+0000 7ff865b86640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff84406a080 0x7ff84406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.002+0000 7ff865b86640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff84406a080 0x7ff84406c540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff8600caa80 tx=0x7ff850007c30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:42.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.006+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7ff85c054420 con 0x7ff8600cd920 2023-10-19T10:52:42.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.191+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8600d3480 con 0x7ff8600cd920 2023-10-19T10:52:42.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.191+0000 7ff856ffd640 1 -- 172.21.15.50:0/207361236 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v55) v1 ==== 80+0+56 (secure 0 0 0) 0x7ff85c057a40 con 0x7ff8600cd920 2023-10-19T10:52:42.195 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.1:0/216420809 2023-10-19T11:46:25.724828+0000 2023-10-19T10:52:42.195 INFO:teuthology.orchestra.run.smithi050.stderr:listed 1 entries 2023-10-19T10:52:42.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff84406a080 msgr2=0x7ff84406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff84406a080 0x7ff84406c540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff8600caa80 tx=0x7ff850007c30 comp rx=0 tx=0).stop 2023-10-19T10:52:42.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 msgr2=0x7ff86015b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff86015b520 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00d800 tx=0x7ff85c00dcc0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 shutdown_connections 2023-10-19T10:52:42.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7ff84406a080 0x7ff84406c540 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff86015c950 0x7ff86015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86015c570 0x7ff86009c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 --2- 172.21.15.50:0/207361236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8600cd920 0x7ff86015b520 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 >> 172.21.15.50:0/207361236 conn(0x7ff860089ff0 msgr2=0x7ff86015bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:42.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 shutdown_connections 2023-10-19T10:52:42.199 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.193+0000 7ff8685eb640 1 -- 172.21.15.50:0/207361236 wait complete. 2023-10-19T10:52:42.207 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi183.front.sepia.ceph.com... 2023-10-19T10:52:42.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:42.208 DEBUG:teuthology.orchestra.run.smithi183:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-19T10:52:42.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi183.stderr:ceph-fuse[95663]: fuse finished with error 0 and tester_r 0 2023-10-19T10:52:42.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:52:42.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:42.241 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-19T10:52:42.409 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:52:42.409 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:42.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055614,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6496528k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19717 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248260k,nr_inodes=812065,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6496528k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-19T10:52:42.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:52:42.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:42.427 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-19T10:52:42.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:52:42.482 INFO:teuthology.orchestra.run.smithi050.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-19T10:52:42.482 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:52:42.483 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:42.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16774 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-19T10:52:42.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:52:42.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:42.501 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-19T10:52:42.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:52:42.558 INFO:teuthology.orchestra.run.smithi183.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-19T10:52:42.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:52:42.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.665+0000 7f93eaebd640 1 Processor -- start 2023-10-19T10:52:42.668 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.665+0000 7f93eaebd640 1 -- start start 2023-10-19T10:52:42.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.665+0000 7f93eaebd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c570 0x7f93e40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.665+0000 7f93eaebd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93e415c950 0x7f93e40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93eaebd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f93e406b760 0x7f93e40d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.670 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93eaebd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e40cde60 con 0x7f93e406b760 2023-10-19T10:52:42.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93eaebd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e406aa30 con 0x7f93e415c950 2023-10-19T10:52:42.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93eaebd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e406b2d0 con 0x7f93e415c570 2023-10-19T10:52:42.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f93e406b760 0x7f93e40d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f93e406b760 0x7f93e40d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56142/0 (socket says 172.21.15.50:56142) 2023-10-19T10:52:42.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 -- 172.21.15.50:0/169471075 learned_addr learned my addr 172.21.15.50:0/169471075 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:42.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e8c59640 1 --2- 172.21.15.50:0/169471075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c570 0x7f93e40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 -- 172.21.15.50:0/169471075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c570 msgr2=0x7f93e40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:42.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 --2- 172.21.15.50:0/169471075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c570 0x7f93e40d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 -- 172.21.15.50:0/169471075 >> v2:172.21.15.183:3300/0 conn(0x7f93e415c950 msgr2=0x7f93e40cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:42.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e3fff640 1 --2- 172.21.15.50:0/169471075 >> v2:172.21.15.183:3300/0 conn(0x7f93e415c950 0x7f93e40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 --2- 172.21.15.50:0/169471075 >> v2:172.21.15.183:3300/0 conn(0x7f93e415c950 0x7f93e40cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e945a640 1 -- 172.21.15.50:0/169471075 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40ce090 con 0x7f93e406b760 2023-10-19T10:52:42.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.666+0000 7f93e3fff640 1 --2- 172.21.15.50:0/169471075 >> v2:172.21.15.183:3300/0 conn(0x7f93e415c950 0x7f93e40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:42.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93e945a640 1 --2- 172.21.15.50:0/169471075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e406b760 0x7f93e40d26a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f93e408a610 tx=0x7f93dc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6229965bab7980 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:42.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93e37fe640 1 -- 172.21.15.50:0/169471075 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc02c9e0 con 0x7f93e406b760 2023-10-19T10:52:42.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93e37fe640 1 -- 172.21.15.50:0/169471075 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc02cb40 con 0x7f93e406b760 2023-10-19T10:52:42.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93e37fe640 1 -- 172.21.15.50:0/169471075 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0343f0 con 0x7f93e406b760 2023-10-19T10:52:42.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93eaebd640 1 -- 172.21.15.50:0/169471075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e406b760 msgr2=0x7f93e40d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93eaebd640 1 --2- 172.21.15.50:0/169471075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e406b760 0x7f93e40d26a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f93e408a610 tx=0x7f93dc02be40 comp rx=0 tx=0).stop 2023-10-19T10:52:42.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93eaebd640 1 -- 172.21.15.50:0/169471075 shutdown_connections 2023-10-19T10:52:42.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93eaebd640 1 --2- 172.21.15.50:0/169471075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e406b760 0x7f93e40d26a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.677 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93eaebd640 1 --2- 172.21.15.50:0/169471075 >> v2:172.21.15.183:3300/0 conn(0x7f93e415c950 0x7f93e40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.667+0000 7f93eaebd640 1 --2- 172.21.15.50:0/169471075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c570 0x7f93e40d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.668+0000 7f93eaebd640 1 -- 172.21.15.50:0/169471075 >> 172.21.15.50:0/169471075 conn(0x7f93e4089ff0 msgr2=0x7f93e409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:42.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.668+0000 7f93eaebd640 1 -- 172.21.15.50:0/169471075 shutdown_connections 2023-10-19T10:52:42.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.668+0000 7f93eaebd640 1 -- 172.21.15.50:0/169471075 wait complete. 2023-10-19T10:52:42.678 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.669+0000 7f93eaebd640 1 Processor -- start 2023-10-19T10:52:42.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.669+0000 7f93eaebd640 1 -- start start 2023-10-19T10:52:42.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.669+0000 7f93eaebd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93e406b760 0x7f93e4152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.669+0000 7f93eaebd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 0x7f93e40cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93eaebd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c950 0x7f93e40cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.679 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93eaebd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b34e0 con 0x7f93e415c570 2023-10-19T10:52:42.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93eaebd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40c6240 con 0x7f93e406b760 2023-10-19T10:52:42.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93eaebd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41527d0 con 0x7f93e415c950 2023-10-19T10:52:42.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 0x7f93e40cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 0x7f93e40cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56146/0 (socket says 172.21.15.50:56146) 2023-10-19T10:52:42.680 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 -- 172.21.15.50:0/1967638986 learned_addr learned my addr 172.21.15.50:0/1967638986 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:42.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e8c59640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93e406b760 0x7f93e4152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e945a640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c950 0x7f93e40cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 -- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c950 msgr2=0x7f93e40cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c950 0x7f93e40cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.681 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 -- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93e406b760 msgr2=0x7f93e4152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93e406b760 0x7f93e4152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 -- 172.21.15.50:0/1967638986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc009b30 con 0x7f93e415c570 2023-10-19T10:52:42.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e3fff640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 0x7f93e40cf450 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c820 tx=0x7f93d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:42.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0007a30 con 0x7f93e415c570 2023-10-19T10:52:42.682 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0007b90 con 0x7f93e415c570 2023-10-19T10:52:42.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e415ee90 con 0x7f93e415c570 2023-10-19T10:52:42.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000ce70 con 0x7f93e415c570 2023-10-19T10:52:42.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.670+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e415f450 con 0x7f93e415c570 2023-10-19T10:52:42.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.671+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e40c63d0 con 0x7f93e415c570 2023-10-19T10:52:42.683 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.672+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f93d00113f0 con 0x7f93e415c570 2023-10-19T10:52:42.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.672+0000 7f93e17fa640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f93c806a050 0x7f93c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:42.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.672+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f93d0088840 con 0x7f93e415c570 2023-10-19T10:52:42.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.673+0000 7f93e8c59640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f93c806a050 0x7f93c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:42.684 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.673+0000 7f93e8c59640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f93c806a050 0x7f93c806c510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93d8005fa0 tx=0x7f93d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:42.685 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.677+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f93d0053e20 con 0x7f93e415c570 2023-10-19T10:52:42.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.887+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93e4073990 con 0x7f93e415c570 2023-10-19T10:52:42.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.887+0000 7f93e17fa640 1 -- 172.21.15.50:0/1967638986 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f93d0057440 con 0x7f93e415c570 2023-10-19T10:52:42.891 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:42.891 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:52:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:52:42.892 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f93c806a050 msgr2=0x7f93c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f93c806a050 0x7f93c806c510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93d8005fa0 tx=0x7f93d8008040 comp rx=0 tx=0).stop 2023-10-19T10:52:42.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 msgr2=0x7f93e40cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:42.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 0x7f93e40cf450 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c820 tx=0x7f93d000cc20 comp rx=0 tx=0).stop 2023-10-19T10:52:42.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 shutdown_connections 2023-10-19T10:52:42.893 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f93c806a050 0x7f93c806c510 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93e415c950 0x7f93e40cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93e415c570 0x7f93e40cf450 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 --2- 172.21.15.50:0/1967638986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93e406b760 0x7f93e4152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:42.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 >> 172.21.15.50:0/1967638986 conn(0x7f93e4089ff0 msgr2=0x7f93e415ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:42.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 shutdown_connections 2023-10-19T10:52:42.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:42.889+0000 7f93eaebd640 1 -- 172.21.15.50:0/1967638986 wait complete. 2023-10-19T10:52:42.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:43.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.006+0000 7effc4399640 1 Processor -- start 2023-10-19T10:52:43.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.006+0000 7effc4399640 1 -- start start 2023-10-19T10:52:43.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc4399640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15a370 0x7effbc06fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc4399640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7effbc0b1f30 0x7effbc0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc4399640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7effbc15de70 0x7effbc15e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc4399640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0b28d0 con 0x7effbc0b1f30 2023-10-19T10:52:43.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc4399640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0caf80 con 0x7effbc15de70 2023-10-19T10:52:43.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc4399640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0d02e0 con 0x7effbc15a370 2023-10-19T10:52:43.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc1934640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7effbc0b1f30 0x7effbc0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc1934640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7effbc0b1f30 0x7effbc0b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56154/0 (socket says 172.21.15.50:56154) 2023-10-19T10:52:43.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.007+0000 7effc1934640 1 -- 172.21.15.50:0/1639208583 learned_addr learned my addr 172.21.15.50:0/1639208583 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:43.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc2936640 1 --2- 172.21.15.50:0/1639208583 >> v2:172.21.15.183:3300/0 conn(0x7effbc15de70 0x7effbc15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc2135640 1 --2- 172.21.15.50:0/1639208583 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15a370 0x7effbc06fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc1934640 1 -- 172.21.15.50:0/1639208583 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15a370 msgr2=0x7effbc06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:43.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc1934640 1 --2- 172.21.15.50:0/1639208583 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15a370 0x7effbc06fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc1934640 1 -- 172.21.15.50:0/1639208583 >> v2:172.21.15.183:3300/0 conn(0x7effbc15de70 msgr2=0x7effbc15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc1934640 1 --2- 172.21.15.50:0/1639208583 >> v2:172.21.15.183:3300/0 conn(0x7effbc15de70 0x7effbc15e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc1934640 1 -- 172.21.15.50:0/1639208583 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc15a750 con 0x7effbc0b1f30 2023-10-19T10:52:43.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.008+0000 7effc1934640 1 --2- 172.21.15.50:0/1639208583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0b2390 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7effbc0b6600 tx=0x7effb002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0bfdd14a63be8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc1133640 1 -- 172.21.15.50:0/1639208583 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0031860 con 0x7effbc0b1f30 2023-10-19T10:52:43.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc1133640 1 -- 172.21.15.50:0/1639208583 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00319c0 con 0x7effbc0b1f30 2023-10-19T10:52:43.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc1133640 1 -- 172.21.15.50:0/1639208583 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0031d50 con 0x7effbc0b1f30 2023-10-19T10:52:43.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 -- 172.21.15.50:0/1639208583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 msgr2=0x7effbc0b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 --2- 172.21.15.50:0/1639208583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0b2390 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7effbc0b6600 tx=0x7effb002b900 comp rx=0 tx=0).stop 2023-10-19T10:52:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 -- 172.21.15.50:0/1639208583 shutdown_connections 2023-10-19T10:52:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 --2- 172.21.15.50:0/1639208583 >> v2:172.21.15.183:3300/0 conn(0x7effbc15de70 0x7effbc15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 --2- 172.21.15.50:0/1639208583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0b2390 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 --2- 172.21.15.50:0/1639208583 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15a370 0x7effbc06fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 -- 172.21.15.50:0/1639208583 >> 172.21.15.50:0/1639208583 conn(0x7effbc0ccf30 msgr2=0x7effbc0cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 -- 172.21.15.50:0/1639208583 shutdown_connections 2023-10-19T10:52:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.009+0000 7effc4399640 1 -- 172.21.15.50:0/1639208583 wait complete. 2023-10-19T10:52:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.010+0000 7effc4399640 1 Processor -- start 2023-10-19T10:52:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.010+0000 7effc4399640 1 -- start start 2023-10-19T10:52:43.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc4399640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc4399640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effbc15a370 0x7effbc153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc4399640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15de70 0x7effbc1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc4399640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0ca9c0 con 0x7effbc0b1f30 2023-10-19T10:52:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc4399640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc14e490 con 0x7effbc15a370 2023-10-19T10:52:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc4399640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc152660 con 0x7effbc15de70 2023-10-19T10:52:43.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56160/0 (socket says 172.21.15.50:56160) 2023-10-19T10:52:43.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 -- 172.21.15.50:0/4228281352 learned_addr learned my addr 172.21.15.50:0/4228281352 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:43.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc1934640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effbc15a370 0x7effbc153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2936640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15de70 0x7effbc1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 -- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15de70 msgr2=0x7effbc1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15de70 0x7effbc1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 -- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effbc15a370 msgr2=0x7effbc153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.011+0000 7effc2135640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effbc15a370 0x7effbc153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effc2135640 1 -- 172.21.15.50:0/4228281352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00094d0 con 0x7effbc0b1f30 2023-10-19T10:52:43.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effc1934640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effbc15a370 0x7effbc153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:43.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effc2135640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0ca480 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7effac007ef0 tx=0x7effac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac014620 con 0x7effbc0b1f30 2023-10-19T10:52:43.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac014780 con 0x7effbc0b1f30 2023-10-19T10:52:43.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac014b30 con 0x7effbc0b1f30 2023-10-19T10:52:43.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc1520e0 con 0x7effbc0b1f30 2023-10-19T10:52:43.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.012+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc1516d0 con 0x7effbc0b1f30 2023-10-19T10:52:43.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.013+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc073990 con 0x7effbc0b1f30 2023-10-19T10:52:43.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.013+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7effac014c90 con 0x7effbc0b1f30 2023-10-19T10:52:43.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.014+0000 7effbaffd640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7effa806a050 0x7effa806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.014+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7effac089130 con 0x7effbc0b1f30 2023-10-19T10:52:43.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.014+0000 7effc1934640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7effa806a050 0x7effa806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.014+0000 7effc1934640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7effa806a050 0x7effa806c510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7effb0036000 tx=0x7effb003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.018+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7effac054670 con 0x7effbc0b1f30 2023-10-19T10:52:43.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.195+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effbc0b2e50 con 0x7effbc0b1f30 2023-10-19T10:52:43.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.196+0000 7effbaffd640 1 -- 172.21.15.50:0/4228281352 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18701 (secure 0 0 0) 0x7effac057c90 con 0x7effbc0b1f30 2023-10-19T10:52:43.199 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:43.199 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:46:26.319171+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:43.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.197+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7effa806a050 msgr2=0x7effa806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7effa806a050 0x7effa806c510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7effb0036000 tx=0x7effb003c040 comp rx=0 tx=0).stop 2023-10-19T10:52:43.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 msgr2=0x7effbc0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0ca480 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7effac007ef0 tx=0x7effac00e4b0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 shutdown_connections 2023-10-19T10:52:43.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7effa806a050 0x7effa806c510 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effbc15de70 0x7effbc1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effbc15a370 0x7effbc153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 --2- 172.21.15.50:0/4228281352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effbc0b1f30 0x7effbc0ca480 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 >> 172.21.15.50:0/4228281352 conn(0x7effbc0ccf30 msgr2=0x7effbc0b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 shutdown_connections 2023-10-19T10:52:43.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.198+0000 7effc4399640 1 -- 172.21.15.50:0/4228281352 wait complete. 2023-10-19T10:52:43.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-19T10:36:42.521036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-19T10:36:48.338828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-19T10:38:26.398890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-19T10:38:26.910608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-19T10:52:43.212 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-19T10:52:43.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.319+0000 7feed3350640 1 Processor -- start 2023-10-19T10:52:43.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.319+0000 7feed3350640 1 -- start start 2023-10-19T10:52:43.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed3350640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feecc0b4070 0x7feecc0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed3350640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15cdf0 0x7feecc0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed3350640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feecc0d3480 con 0x7feecc15c960 2023-10-19T10:52:43.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed3350640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feecc0b4990 con 0x7feecc0b4070 2023-10-19T10:52:43.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed3350640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc15d280 con 0x7feecc15cdf0 2023-10-19T10:52:43.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed18ed640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15cdf0 0x7feecc0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed10ec640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feecc0b4070 0x7feecc0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed08eb640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7feecc15c960 0x7feecc0d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:32816/0 (socket says 172.21.15.50:32816) 2023-10-19T10:52:43.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.320+0000 7feed08eb640 1 -- 172.21.15.50:0/3451130625 learned_addr learned my addr 172.21.15.50:0/3451130625 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:43.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feed10ec640 1 -- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15cdf0 msgr2=0x7feecc0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:43.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feed10ec640 1 --2- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15cdf0 0x7feecc0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feed10ec640 1 -- 172.21.15.50:0/3451130625 >> v1:172.21.15.183:6789/0 conn(0x7feecc15c960 legacy=0x7feecc0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feed10ec640 1 -- 172.21.15.50:0/3451130625 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc15d560 con 0x7feecc0b4070 2023-10-19T10:52:43.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feec3fff640 1 -- 172.21.15.50:0/3451130625 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691987366 0 0) 0x7feecc0d3480 con 0x7feecc15c960 2023-10-19T10:52:43.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feed18ed640 1 --2- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15cdf0 0x7feecc0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:43.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.321+0000 7feed10ec640 1 --2- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc0b4450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feec8009de0 tx=0x7feec802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3bfb3a5ae7fa36 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feec3fff640 1 -- 172.21.15.50:0/3451130625 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec802cc90 con 0x7feecc0b4070 2023-10-19T10:52:43.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feec3fff640 1 -- 172.21.15.50:0/3451130625 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec802cdf0 con 0x7feecc0b4070 2023-10-19T10:52:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feec3fff640 1 -- 172.21.15.50:0/3451130625 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8034660 con 0x7feecc0b4070 2023-10-19T10:52:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 -- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 msgr2=0x7feecc0b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 --2- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc0b4450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feec8009de0 tx=0x7feec802bee0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 -- 172.21.15.50:0/3451130625 shutdown_connections 2023-10-19T10:52:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 --2- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15cdf0 0x7feecc0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 --2- 172.21.15.50:0/3451130625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc0b4450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 -- 172.21.15.50:0/3451130625 >> 172.21.15.50:0/3451130625 conn(0x7feecc09b970 msgr2=0x7feecc0d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 -- 172.21.15.50:0/3451130625 shutdown_connections 2023-10-19T10:52:43.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.322+0000 7feed3350640 1 -- 172.21.15.50:0/3451130625 wait complete. 2023-10-19T10:52:43.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.323+0000 7feed3350640 1 Processor -- start 2023-10-19T10:52:43.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.323+0000 7feed3350640 1 -- start start 2023-10-19T10:52:43.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed3350640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed3350640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 0x7feecc0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed3350640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15fc60 0x7feecc0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed3350640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0cabb0 con 0x7feecc15c960 2023-10-19T10:52:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed3350640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc151020 con 0x7feecc0b4070 2023-10-19T10:52:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed3350640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc0c9fa0 con 0x7feecc15fc60 2023-10-19T10:52:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed08eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 0x7feecc0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed08eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 0x7feecc0c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56162/0 (socket says 172.21.15.50:56162) 2023-10-19T10:52:43.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed08eb640 1 -- 172.21.15.50:0/3759355327 learned_addr learned my addr 172.21.15.50:0/3759355327 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:43.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed18ed640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15fc60 0x7feecc0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.324+0000 7feed10ec640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed08eb640 1 -- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15fc60 msgr2=0x7feecc0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed08eb640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15fc60 0x7feecc0b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed08eb640 1 -- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 msgr2=0x7feecc150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed08eb640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed08eb640 1 -- 172.21.15.50:0/3759355327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4009400 con 0x7feecc15c960 2023-10-19T10:52:43.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed10ec640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:43.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed08eb640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 0x7feecc0c9630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c4a0 tx=0x7feebc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc005ce0 con 0x7feecc15c960 2023-10-19T10:52:43.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc005e40 con 0x7feecc15c960 2023-10-19T10:52:43.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec8009a70 con 0x7feecc15c960 2023-10-19T10:52:43.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0072b0 con 0x7feecc15c960 2023-10-19T10:52:43.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.325+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc0ca560 con 0x7feecc15c960 2023-10-19T10:52:43.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.326+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feecc04e420 con 0x7feecc15c960 2023-10-19T10:52:43.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.327+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feebc007410 con 0x7feecc15c960 2023-10-19T10:52:43.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.327+0000 7feec1ffb640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feeb006a050 0x7feeb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.327+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7feebc087d30 con 0x7feecc15c960 2023-10-19T10:52:43.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.327+0000 7feed10ec640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feeb006a050 0x7feeb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.328+0000 7feed10ec640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feeb006a050 0x7feeb006c510 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feec8036000 tx=0x7feec8030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.331+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7feebc053270 con 0x7feecc15c960 2023-10-19T10:52:43.510 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.507+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feecc08c720 con 0x7feecc15c960 2023-10-19T10:52:43.511 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.508+0000 7feec1ffb640 1 -- 172.21.15.50:0/3759355327 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7feebc0170a0 con 0x7feecc15c960 2023-10-19T10:52:43.511 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2023-10-19T10:52:43.512 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.509+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feeb006a050 msgr2=0x7feeb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.509+0000 7feed3350640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feeb006a050 0x7feeb006c510 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feec8036000 tx=0x7feec8030000 comp rx=0 tx=0).stop 2023-10-19T10:52:43.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 msgr2=0x7feecc0c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 0x7feecc0c9630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c4a0 tx=0x7feebc00c960 comp rx=0 tx=0).stop 2023-10-19T10:52:43.513 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 shutdown_connections 2023-10-19T10:52:43.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7feeb006a050 0x7feeb006c510 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feecc15fc60 0x7feecc0b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feecc15c960 0x7feecc0c9630 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 --2- 172.21.15.50:0/3759355327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc0b4070 0x7feecc150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 >> 172.21.15.50:0/3759355327 conn(0x7feecc09b970 msgr2=0x7feecc0b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 shutdown_connections 2023-10-19T10:52:43.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.510+0000 7feed3350640 1 -- 172.21.15.50:0/3759355327 wait complete. 2023-10-19T10:52:43.523 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-19T10:52:43.633 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.630+0000 7f300e1c7640 1 Processor -- start 2023-10-19T10:52:43.634 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.630+0000 7f300e1c7640 1 -- start start 2023-10-19T10:52:43.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f300e1c7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f30080b4070 0x7f30080b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f300e1c7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f300815c960 0x7f30080d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f300e1c7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f300815cdf0 0x7f3008160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.636 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f300e1c7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30080b4990 con 0x7f30080b4070 2023-10-19T10:52:43.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f300e1c7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300815d280 con 0x7f300815cdf0 2023-10-19T10:52:43.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f300e1c7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300815d560 con 0x7f300815c960 2023-10-19T10:52:43.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f30080b4070 0x7f30080b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f30080b4070 0x7f30080b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56166/0 (socket says 172.21.15.50:56166) 2023-10-19T10:52:43.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 -- 172.21.15.50:0/4072017801 learned_addr learned my addr 172.21.15.50:0/4072017801 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:43.637 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f3006ffd640 1 --2- 172.21.15.50:0/4072017801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f300815c960 0x7f30080d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f3007fff640 1 --2- 172.21.15.50:0/4072017801 >> v2:172.21.15.183:3300/0 conn(0x7f300815cdf0 0x7f3008160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 -- 172.21.15.50:0/4072017801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f300815c960 msgr2=0x7f30080d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:43.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 --2- 172.21.15.50:0/4072017801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f300815c960 0x7f30080d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 -- 172.21.15.50:0/4072017801 >> v2:172.21.15.183:3300/0 conn(0x7f300815cdf0 msgr2=0x7f3008160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:43.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 --2- 172.21.15.50:0/4072017801 >> v2:172.21.15.183:3300/0 conn(0x7f300815cdf0 0x7f3008160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.639 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.631+0000 7f30077fe640 1 -- 172.21.15.50:0/4072017801 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300815d840 con 0x7f30080b4070 2023-10-19T10:52:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.632+0000 7f30077fe640 1 --2- 172.21.15.50:0/4072017801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080b4070 0x7f30080b4450 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f30081601c0 tx=0x7f2ff802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4d7cbe753f4f17 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.632+0000 7f30067fc640 1 -- 172.21.15.50:0/4072017801 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004230 con 0x7f30080b4070 2023-10-19T10:52:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.632+0000 7f30067fc640 1 -- 172.21.15.50:0/4072017801 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff8004390 con 0x7f30080b4070 2023-10-19T10:52:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.632+0000 7f30067fc640 1 -- 172.21.15.50:0/4072017801 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8003810 con 0x7f30080b4070 2023-10-19T10:52:43.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 -- 172.21.15.50:0/4072017801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080b4070 msgr2=0x7f30080b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 --2- 172.21.15.50:0/4072017801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080b4070 0x7f30080b4450 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f30081601c0 tx=0x7f2ff802b8f0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 -- 172.21.15.50:0/4072017801 shutdown_connections 2023-10-19T10:52:43.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 --2- 172.21.15.50:0/4072017801 >> v2:172.21.15.183:3300/0 conn(0x7f300815cdf0 0x7f3008160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 --2- 172.21.15.50:0/4072017801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f300815c960 0x7f30080d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 --2- 172.21.15.50:0/4072017801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080b4070 0x7f30080b4450 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 -- 172.21.15.50:0/4072017801 >> 172.21.15.50:0/4072017801 conn(0x7f300809b970 msgr2=0x7f30080d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 -- 172.21.15.50:0/4072017801 shutdown_connections 2023-10-19T10:52:43.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.633+0000 7f300e1c7640 1 -- 172.21.15.50:0/4072017801 wait complete. 2023-10-19T10:52:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.634+0000 7f300e1c7640 1 Processor -- start 2023-10-19T10:52:43.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.634+0000 7f300e1c7640 1 -- start start 2023-10-19T10:52:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f300e1c7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30080b4070 0x7f30080ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f300e1c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300815c960 0x7f30080c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f300e1c7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 0x7f30080ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f300e1c7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30080b5fc0 con 0x7f30080cc430 2023-10-19T10:52:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f300e1c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30080b53b0 con 0x7f300815c960 2023-10-19T10:52:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f300e1c7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30080b5690 con 0x7f30080b4070 2023-10-19T10:52:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 0x7f30080ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 0x7f30080ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56174/0 (socket says 172.21.15.50:56174) 2023-10-19T10:52:43.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 -- 172.21.15.50:0/2666529876 learned_addr learned my addr 172.21.15.50:0/2666529876 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:43.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f30077fe640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30080b4070 0x7f30080ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.648 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3006ffd640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300815c960 0x7f30080c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 -- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30080b4070 msgr2=0x7f30080ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30080b4070 0x7f30080ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 -- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300815c960 msgr2=0x7f30080c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300815c960 0x7f30080c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 -- 172.21.15.50:0/2666529876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8009460 con 0x7f30080cc430 2023-10-19T10:52:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3006ffd640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300815c960 0x7f30080c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.635+0000 7f3007fff640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 0x7f30080ca530 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3000009fd0 tx=0x7f300000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.636+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000014420 con 0x7f30080cc430 2023-10-19T10:52:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.636+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000014580 con 0x7f30080cc430 2023-10-19T10:52:43.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.636+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30080b5970 con 0x7f30080cc430 2023-10-19T10:52:43.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.636+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000014930 con 0x7f30080cc430 2023-10-19T10:52:43.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.636+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30080ce6b0 con 0x7f30080cc430 2023-10-19T10:52:43.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.637+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30080b6130 con 0x7f30080cc430 2023-10-19T10:52:43.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.637+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3000014a90 con 0x7f30080cc430 2023-10-19T10:52:43.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.637+0000 7f2feffff640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ff406a050 0x7f2ff406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.638+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3000088fe0 con 0x7f30080cc430 2023-10-19T10:52:43.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.638+0000 7f30077fe640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ff406a050 0x7f2ff406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.638+0000 7f30077fe640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ff406a050 0x7f2ff406c510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8035000 tx=0x7f2ff8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.642+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3000057f70 con 0x7f30080cc430 2023-10-19T10:52:43.818 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.815+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f300804e420 con 0x7f30080cc430 2023-10-19T10:52:43.819 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.816+0000 7f2feffff640 1 -- 172.21.15.50:0/2666529876 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3000057b00 con 0x7f30080cc430 2023-10-19T10:52:43.819 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2023-10-19T10:52:43.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ff406a050 msgr2=0x7f2ff406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ff406a050 0x7f2ff406c510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8035000 tx=0x7f2ff8008030 comp rx=0 tx=0).stop 2023-10-19T10:52:43.821 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 msgr2=0x7f30080ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 0x7f30080ca530 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3000009fd0 tx=0x7f300000e4b0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 shutdown_connections 2023-10-19T10:52:43.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2ff406a050 0x7f2ff406c510 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.822 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30080cc430 0x7f30080ca530 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300815c960 0x7f30080c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 --2- 172.21.15.50:0/2666529876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f30080b4070 0x7f30080ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 >> 172.21.15.50:0/2666529876 conn(0x7f300809b970 msgr2=0x7f3008160210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 shutdown_connections 2023-10-19T10:52:43.823 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.818+0000 7f300e1c7640 1 -- 172.21.15.50:0/2666529876 wait complete. 2023-10-19T10:52:43.832 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-19T10:52:43.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 Processor -- start 2023-10-19T10:52:43.942 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 -- start start 2023-10-19T10:52:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe6180b4070 0x7fe6180b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe61815cdf0 0x7fe6180cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180d3480 con 0x7fe61815c960 2023-10-19T10:52:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180b4990 con 0x7fe61815cdf0 2023-10-19T10:52:43.944 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.939+0000 7fe61d3d2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61815d280 con 0x7fe6180b4070 2023-10-19T10:52:43.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe6177fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe61815cdf0 0x7fe6180cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe60e7fc640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe61815c960 0x7fe6180d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50088/0 (socket says 172.21.15.50:50088) 2023-10-19T10:52:43.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe60e7fc640 1 -- 172.21.15.50:0/1750141878 learned_addr learned my addr 172.21.15.50:0/1750141878 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:43.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe616ffd640 1 --2- 172.21.15.50:0/1750141878 >> v2:172.21.15.183:3300/0 conn(0x7fe6180b4070 0x7fe6180b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.945 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe6177fe640 1 -- 172.21.15.50:0/1750141878 >> v2:172.21.15.183:3300/0 conn(0x7fe6180b4070 msgr2=0x7fe6180b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe6177fe640 1 --2- 172.21.15.50:0/1750141878 >> v2:172.21.15.183:3300/0 conn(0x7fe6180b4070 0x7fe6180b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe6177fe640 1 -- 172.21.15.50:0/1750141878 >> v1:172.21.15.50:6789/0 conn(0x7fe61815c960 legacy=0x7fe6180d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe6167fc640 1 -- 172.21.15.50:0/1750141878 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695819895 0 0) 0x7fe6180d3480 con 0x7fe61815c960 2023-10-19T10:52:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe6177fe640 1 -- 172.21.15.50:0/1750141878 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61815d560 con 0x7fe61815cdf0 2023-10-19T10:52:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.940+0000 7fe616ffd640 1 --2- 172.21.15.50:0/1750141878 >> v2:172.21.15.183:3300/0 conn(0x7fe6180b4070 0x7fe6180b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe6177fe640 1 --2- 172.21.15.50:0/1750141878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61815cdf0 0x7fe6180cd4b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe61808a3b0 tx=0x7fe61002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d174795e25c99682 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe6167fc640 1 -- 172.21.15.50:0/1750141878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61002cc80 con 0x7fe61815cdf0 2023-10-19T10:52:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe6167fc640 1 -- 172.21.15.50:0/1750141878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61002cde0 con 0x7fe61815cdf0 2023-10-19T10:52:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe6167fc640 1 -- 172.21.15.50:0/1750141878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610034660 con 0x7fe61815cdf0 2023-10-19T10:52:43.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 -- 172.21.15.50:0/1750141878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61815cdf0 msgr2=0x7fe6180cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.950 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/1750141878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61815cdf0 0x7fe6180cd4b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe61808a3b0 tx=0x7fe61002bed0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 -- 172.21.15.50:0/1750141878 shutdown_connections 2023-10-19T10:52:43.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/1750141878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61815cdf0 0x7fe6180cd4b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/1750141878 >> v2:172.21.15.183:3300/0 conn(0x7fe6180b4070 0x7fe6180b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.951 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 -- 172.21.15.50:0/1750141878 >> 172.21.15.50:0/1750141878 conn(0x7fe61809b970 msgr2=0x7fe6180d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 -- 172.21.15.50:0/1750141878 shutdown_connections 2023-10-19T10:52:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.941+0000 7fe61d3d2640 1 -- 172.21.15.50:0/1750141878 wait complete. 2023-10-19T10:52:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.942+0000 7fe61d3d2640 1 Processor -- start 2023-10-19T10:52:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 -- start start 2023-10-19T10:52:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6180b4070 0x7fe6181555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe61815c960 0x7fe618154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 0x7fe6180c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180cb960 con 0x7fe6180cb110 2023-10-19T10:52:43.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c7530 con 0x7fe61815c960 2023-10-19T10:52:43.953 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe61d3d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c6c40 con 0x7fe6180b4070 2023-10-19T10:52:43.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.943+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 0x7fe6180c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 0x7fe6180c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56200/0 (socket says 172.21.15.50:56200) 2023-10-19T10:52:43.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 -- 172.21.15.50:0/599428632 learned_addr learned my addr 172.21.15.50:0/599428632 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:43.954 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe60e7fc640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe61815c960 0x7fe618154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe616ffd640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6180b4070 0x7fe6181555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 -- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6180b4070 msgr2=0x7fe6181555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6180b4070 0x7fe6181555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.955 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 -- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe61815c960 msgr2=0x7fe618154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:43.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe61815c960 0x7fe618154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:43.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 -- 172.21.15.50:0/599428632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610009a70 con 0x7fe6180cb110 2023-10-19T10:52:43.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe616ffd640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6180b4070 0x7fe6181555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:43.956 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe60e7fc640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe61815c960 0x7fe618154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe6177fe640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 0x7fe6180c79c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe610009e80 tx=0x7fe610004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610040650 con 0x7fe6180cb110 2023-10-19T10:52:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6100407b0 con 0x7fe6180cb110 2023-10-19T10:52:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe618155390 con 0x7fe6180cb110 2023-10-19T10:52:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610040b60 con 0x7fe6180cb110 2023-10-19T10:52:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.944+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6180c71c0 con 0x7fe6180cb110 2023-10-19T10:52:43.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.945+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61804e420 con 0x7fe6180cb110 2023-10-19T10:52:43.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.946+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe610040cc0 con 0x7fe6180cb110 2023-10-19T10:52:43.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.946+0000 7fe60ffff640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe5f806a050 0x7fe5f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:43.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.946+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fe6100ab980 con 0x7fe6180cb110 2023-10-19T10:52:43.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.946+0000 7fe616ffd640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe5f806a050 0x7fe5f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.947+0000 7fe616ffd640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe5f806a050 0x7fe5f806c510 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe618156170 tx=0x7fe608008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:43.950+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fe610077130 con 0x7fe6180cb110 2023-10-19T10:52:44.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.123+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6180c7670 con 0x7fe6180cb110 2023-10-19T10:52:44.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.123+0000 7fe60ffff640 1 -- 172.21.15.50:0/599428632 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe61007a750 con 0x7fe6180cb110 2023-10-19T10:52:44.127 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2023-10-19T10:52:44.128 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe5f806a050 msgr2=0x7fe5f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe5f806a050 0x7fe5f806c510 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe618156170 tx=0x7fe608008040 comp rx=0 tx=0).stop 2023-10-19T10:52:44.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 msgr2=0x7fe6180c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 0x7fe6180c79c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe610009e80 tx=0x7fe610004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:44.129 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 shutdown_connections 2023-10-19T10:52:44.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fe5f806a050 0x7fe5f806c510 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6180cb110 0x7fe6180c79c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe61815c960 0x7fe618154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.130 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 --2- 172.21.15.50:0/599428632 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6180b4070 0x7fe6181555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 >> 172.21.15.50:0/599428632 conn(0x7fe61809b970 msgr2=0x7fe6180b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:44.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 shutdown_connections 2023-10-19T10:52:44.131 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.125+0000 7fe61d3d2640 1 -- 172.21.15.50:0/599428632 wait complete. 2023-10-19T10:52:44.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-19T10:52:44.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.245+0000 7f37cce48640 1 Processor -- start 2023-10-19T10:52:44.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.245+0000 7f37cce48640 1 -- start start 2023-10-19T10:52:44.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.246+0000 7f37cce48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 0x7f37c806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.246+0000 7f37cce48640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37c80ce730 0x7f37c806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.246+0000 7f37cce48640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c806f2a0 con 0x7f37c815c6f0 2023-10-19T10:52:44.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.246+0000 7f37cce48640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c806fe20 con 0x7f37c80ce730 2023-10-19T10:52:44.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.246+0000 7f37cce48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80702b0 con 0x7f37c80b1f60 2023-10-19T10:52:44.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 0x7f37c806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 0x7f37c806ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:58160/0 (socket says 172.21.15.50:58160) 2023-10-19T10:52:44.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 -- 172.21.15.50:0/3576743677 learned_addr learned my addr 172.21.15.50:0/3576743677 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:44.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c6d9d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37c80ce730 0x7f37c806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 -- 172.21.15.50:0/3576743677 >> v2:172.21.15.183:3300/0 conn(0x7f37c80ce730 msgr2=0x7f37c806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:44.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 --2- 172.21.15.50:0/3576743677 >> v2:172.21.15.183:3300/0 conn(0x7f37c80ce730 0x7f37c806b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 -- 172.21.15.50:0/3576743677 >> v1:172.21.15.183:6789/0 conn(0x7f37c815c6f0 legacy=0x7f37c80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.247+0000 7f37c5d9b640 1 -- 172.21.15.50:0/3576743677 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c806f4f0 con 0x7f37c80b1f60 2023-10-19T10:52:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.248+0000 7f37c5d9b640 1 --2- 172.21.15.50:0/3576743677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 0x7f37c806ab20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009870 tx=0x7f37b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0cfab4140ad7c86 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.248+0000 7f37c559a640 1 -- 172.21.15.50:0/3576743677 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0030840 con 0x7f37c80b1f60 2023-10-19T10:52:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.248+0000 7f37c559a640 1 -- 172.21.15.50:0/3576743677 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b00309a0 con 0x7f37c80b1f60 2023-10-19T10:52:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.248+0000 7f37c559a640 1 -- 172.21.15.50:0/3576743677 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0030d30 con 0x7f37c80b1f60 2023-10-19T10:52:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.248+0000 7f37cce48640 1 -- 172.21.15.50:0/3576743677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 msgr2=0x7f37c806ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.248+0000 7f37cce48640 1 --2- 172.21.15.50:0/3576743677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 0x7f37c806ab20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009870 tx=0x7f37b002b980 comp rx=0 tx=0).stop 2023-10-19T10:52:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.249+0000 7f37cce48640 1 -- 172.21.15.50:0/3576743677 shutdown_connections 2023-10-19T10:52:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.249+0000 7f37cce48640 1 --2- 172.21.15.50:0/3576743677 >> v2:172.21.15.183:3300/0 conn(0x7f37c80ce730 0x7f37c806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.249+0000 7f37cce48640 1 --2- 172.21.15.50:0/3576743677 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b1f60 0x7f37c806ab20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.249+0000 7f37cce48640 1 -- 172.21.15.50:0/3576743677 >> 172.21.15.50:0/3576743677 conn(0x7f37c80ccf60 msgr2=0x7f37c80cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.249+0000 7f37cce48640 1 -- 172.21.15.50:0/3576743677 shutdown_connections 2023-10-19T10:52:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.249+0000 7f37cce48640 1 -- 172.21.15.50:0/3576743677 wait complete. 2023-10-19T10:52:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.250+0000 7f37cce48640 1 Processor -- start 2023-10-19T10:52:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.250+0000 7f37cce48640 1 -- start start 2023-10-19T10:52:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37cce48640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80ce730 0x7f37c814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37cce48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c815c6f0 0x7f37c815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37cce48640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 0x7f37c80c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37cce48640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80b5960 con 0x7f37c80b4ec0 2023-10-19T10:52:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37cce48640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8159910 con 0x7f37c80ce730 2023-10-19T10:52:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37cce48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80b53a0 con 0x7f37c815c6f0 2023-10-19T10:52:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 0x7f37c80c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 0x7f37c80c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56204/0 (socket says 172.21.15.50:56204) 2023-10-19T10:52:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 -- 172.21.15.50:0/2484583430 learned_addr learned my addr 172.21.15.50:0/2484583430 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c5d9b640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c815c6f0 0x7f37c815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c659c640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80ce730 0x7f37c814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 -- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c815c6f0 msgr2=0x7f37c815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c815c6f0 0x7f37c815b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 -- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80ce730 msgr2=0x7f37c814ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80ce730 0x7f37c814ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 -- 172.21.15.50:0/2484583430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00094d0 con 0x7f37c80b4ec0 2023-10-19T10:52:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.251+0000 7f37c6d9d640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 0x7f37c80c45a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f37b800efc0 tx=0x7f37b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.252+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8009ca0 con 0x7f37c80b4ec0 2023-10-19T10:52:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.252+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800f040 con 0x7f37c80b4ec0 2023-10-19T10:52:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.252+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c80b5680 con 0x7f37c80b4ec0 2023-10-19T10:52:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.252+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b80146c0 con 0x7f37c80b4ec0 2023-10-19T10:52:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.252+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c814d360 con 0x7f37c80b4ec0 2023-10-19T10:52:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.253+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c804e420 con 0x7f37c80b4ec0 2023-10-19T10:52:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.253+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f37b8014820 con 0x7f37c80b4ec0 2023-10-19T10:52:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.253+0000 7f37beffd640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a806a050 0x7f37a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.254+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f37b8089540 con 0x7f37c80b4ec0 2023-10-19T10:52:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.254+0000 7f37c659c640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a806a050 0x7f37a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.254+0000 7f37c659c640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a806a050 0x7f37a806c510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f37c80c71b0 tx=0x7f37c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.258+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f37b8054b30 con 0x7f37c80b4ec0 2023-10-19T10:52:44.435 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.432+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37c8159a50 con 0x7f37c80b4ec0 2023-10-19T10:52:44.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.432+0000 7f37beffd640 1 -- 172.21.15.50:0/2484583430 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37b8058150 con 0x7f37c80b4ec0 2023-10-19T10:52:44.436 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2023-10-19T10:52:44.437 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a806a050 msgr2=0x7f37a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a806a050 0x7f37a806c510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f37c80c71b0 tx=0x7f37c0009040 comp rx=0 tx=0).stop 2023-10-19T10:52:44.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 msgr2=0x7f37c80c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 0x7f37c80c45a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f37b800efc0 tx=0x7f37b800c490 comp rx=0 tx=0).stop 2023-10-19T10:52:44.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 shutdown_connections 2023-10-19T10:52:44.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f37a806a050 0x7f37a806c510 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37c80b4ec0 0x7f37c80c45a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c815c6f0 0x7f37c815b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 --2- 172.21.15.50:0/2484583430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80ce730 0x7f37c814ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 >> 172.21.15.50:0/2484583430 conn(0x7f37c80ccf60 msgr2=0x7f37c80b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:44.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 shutdown_connections 2023-10-19T10:52:44.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.434+0000 7f37cce48640 1 -- 172.21.15.50:0/2484583430 wait complete. 2023-10-19T10:52:44.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:52:44.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.552+0000 7f7af6477640 1 Processor -- start 2023-10-19T10:52:44.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.552+0000 7f7af6477640 1 -- start start 2023-10-19T10:52:44.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.552+0000 7f7af6477640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7af015c570 0x7f7af00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.556 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.552+0000 7f7af6477640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7af015c950 0x7f7af00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.552+0000 7f7af6477640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af00cde60 con 0x7f7af006b760 2023-10-19T10:52:44.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7af6477640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af006aa30 con 0x7f7af015c950 2023-10-19T10:52:44.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7af6477640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af006b2d0 con 0x7f7af015c570 2023-10-19T10:52:44.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7aef7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7af015c950 0x7f7af00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7af4a14640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f7af006b760 0x7f7af00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50092/0 (socket says 172.21.15.50:50092) 2023-10-19T10:52:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7af4a14640 1 -- 172.21.15.50:0/1655416721 learned_addr learned my addr 172.21.15.50:0/1655416721 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7aeffff640 1 --2- 172.21.15.50:0/1655416721 >> v2:172.21.15.183:3300/0 conn(0x7f7af015c570 0x7f7af00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7aef7fe640 1 -- 172.21.15.50:0/1655416721 >> v2:172.21.15.183:3300/0 conn(0x7f7af015c570 msgr2=0x7f7af00d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7aef7fe640 1 --2- 172.21.15.50:0/1655416721 >> v2:172.21.15.183:3300/0 conn(0x7f7af015c570 0x7f7af00d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7aef7fe640 1 -- 172.21.15.50:0/1655416721 >> v1:172.21.15.50:6789/0 conn(0x7f7af006b760 legacy=0x7f7af00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.553+0000 7f7aef7fe640 1 -- 172.21.15.50:0/1655416721 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af00ce090 con 0x7f7af015c950 2023-10-19T10:52:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7aef7fe640 1 --2- 172.21.15.50:0/1655416721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af00cd920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7af00d2670 tx=0x7f7ae002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b0ae31fa45158 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7aeeffd640 1 -- 172.21.15.50:0/1655416721 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae002e070 con 0x7f7af015c950 2023-10-19T10:52:44.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7aeeffd640 1 -- 172.21.15.50:0/1655416721 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae00315a0 con 0x7f7af015c950 2023-10-19T10:52:44.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7aeeffd640 1 -- 172.21.15.50:0/1655416721 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0031930 con 0x7f7af015c950 2023-10-19T10:52:44.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 -- 172.21.15.50:0/1655416721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 msgr2=0x7f7af00cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 --2- 172.21.15.50:0/1655416721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af00cd920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7af00d2670 tx=0x7f7ae002b840 comp rx=0 tx=0).stop 2023-10-19T10:52:44.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 -- 172.21.15.50:0/1655416721 shutdown_connections 2023-10-19T10:52:44.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 --2- 172.21.15.50:0/1655416721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af00cd920 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 --2- 172.21.15.50:0/1655416721 >> v2:172.21.15.183:3300/0 conn(0x7f7af015c570 0x7f7af00d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 -- 172.21.15.50:0/1655416721 >> 172.21.15.50:0/1655416721 conn(0x7f7af0089ff0 msgr2=0x7f7af009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:44.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.554+0000 7f7af6477640 1 -- 172.21.15.50:0/1655416721 shutdown_connections 2023-10-19T10:52:44.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.555+0000 7f7af6477640 1 -- 172.21.15.50:0/1655416721 wait complete. 2023-10-19T10:52:44.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 Processor -- start 2023-10-19T10:52:44.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 -- start start 2023-10-19T10:52:44.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7af006b760 0x7f7af0154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7af015c570 0x7f7af00b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0146640 con 0x7f7af015c950 2023-10-19T10:52:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0146920 con 0x7f7af006b760 2023-10-19T10:52:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.556+0000 7f7af6477640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0146c00 con 0x7f7af015c570 2023-10-19T10:52:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56224/0 (socket says 172.21.15.50:56224) 2023-10-19T10:52:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 -- 172.21.15.50:0/2801646884 learned_addr learned my addr 172.21.15.50:0/2801646884 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7aef7fe640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7af015c570 0x7f7af00b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7aeffff640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7af006b760 0x7f7af0154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 -- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7af015c570 msgr2=0x7f7af00b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7af015c570 0x7f7af00b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 -- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7af006b760 msgr2=0x7f7af0154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7af006b760 0x7f7af0154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 -- 172.21.15.50:0/2801646884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009400 con 0x7f7af015c950 2023-10-19T10:52:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af4a14640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af015b920 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800caa0 tx=0x7f7ae800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800ab40 con 0x7f7af015c950 2023-10-19T10:52:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae800aca0 con 0x7f7af015c950 2023-10-19T10:52:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af0146ee0 con 0x7f7af015c950 2023-10-19T10:52:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8012600 con 0x7f7af015c950 2023-10-19T10:52:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.557+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01552d0 con 0x7f7af015c950 2023-10-19T10:52:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.558+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af0147400 con 0x7f7af015c950 2023-10-19T10:52:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.559+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ae8007070 con 0x7f7af015c950 2023-10-19T10:52:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.559+0000 7f7aecff9640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7adc06a050 0x7f7adc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.559+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f7ae8088500 con 0x7f7af015c950 2023-10-19T10:52:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.560+0000 7f7aeffff640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7adc06a050 0x7f7adc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.560+0000 7f7aeffff640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7adc06a050 0x7f7adc06c510 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006a30 tx=0x7f7ae4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.564+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f7af0147400 con 0x7f7af015c950 2023-10-19T10:52:44.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.779+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7af0147400 con 0x7f7af015c950 2023-10-19T10:52:44.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.779+0000 7f7aecff9640 1 -- 172.21.15.50:0/2801646884 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f7af0147400 con 0x7f7af015c950 2023-10-19T10:52:44.783 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:44.783 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:52:44.784 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:52:44.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7adc06a050 msgr2=0x7f7adc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7adc06a050 0x7f7adc06c510 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006a30 tx=0x7f7ae4008040 comp rx=0 tx=0).stop 2023-10-19T10:52:44.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 msgr2=0x7f7af015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af015b920 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800caa0 tx=0x7f7ae800cf60 comp rx=0 tx=0).stop 2023-10-19T10:52:44.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 shutdown_connections 2023-10-19T10:52:44.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f7adc06a050 0x7f7adc06c510 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af015c950 0x7f7af015b920 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7af015c570 0x7f7af00b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 --2- 172.21.15.50:0/2801646884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7af006b760 0x7f7af0154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 >> 172.21.15.50:0/2801646884 conn(0x7f7af0089ff0 msgr2=0x7f7af00b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:44.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.781+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 shutdown_connections 2023-10-19T10:52:44.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.782+0000 7f7af6477640 1 -- 172.21.15.50:0/2801646884 wait complete. 2023-10-19T10:52:44.796 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:44.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.899+0000 7fae4d1de640 1 Processor -- start 2023-10-19T10:52:44.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4d1de640 1 -- start start 2023-10-19T10:52:44.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4d1de640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fae4815a370 0x7fae4806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4d1de640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480caf80 con 0x7fae480b1f30 2023-10-19T10:52:44.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4d1de640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480d02e0 con 0x7fae480b23c0 2023-10-19T10:52:44.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4d1de640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4815ec90 con 0x7fae4815a370 2023-10-19T10:52:44.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae46d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fae4815a370 0x7fae4806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4659c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fae480b1f30 0x7fae4815de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50098/0 (socket says 172.21.15.50:50098) 2023-10-19T10:52:44.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae46d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fae4815a370 0x7fae4806fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56234/0 (socket says 172.21.15.50:56234) 2023-10-19T10:52:44.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.900+0000 7fae4659c640 1 -- 172.21.15.50:0/495862485 learned_addr learned my addr 172.21.15.50:0/495862485 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:44.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.901+0000 7fae46d9d640 1 -- 172.21.15.50:0/495862485 >> v1:172.21.15.183:6789/0 conn(0x7fae480b23c0 legacy=0x7fae4815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:44.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.901+0000 7fae46d9d640 1 -- 172.21.15.50:0/495862485 >> v1:172.21.15.50:6789/0 conn(0x7fae480b1f30 legacy=0x7fae4815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.901+0000 7fae46d9d640 1 -- 172.21.15.50:0/495862485 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae480b2850 con 0x7fae4815a370 2023-10-19T10:52:44.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.901+0000 7fae45d9b640 1 -- 172.21.15.50:0/495862485 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752272796 0 0) 0x7fae480caf80 con 0x7fae480b1f30 2023-10-19T10:52:44.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.901+0000 7fae46d9d640 1 --2- 172.21.15.50:0/495862485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae4806fc80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d90 tx=0x7fae3802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c655c369b7ccd4ed server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae45d9b640 1 -- 172.21.15.50:0/495862485 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3802cc60 con 0x7fae4815a370 2023-10-19T10:52:44.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae45d9b640 1 -- 172.21.15.50:0/495862485 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3802cdc0 con 0x7fae4815a370 2023-10-19T10:52:44.907 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae45d9b640 1 -- 172.21.15.50:0/495862485 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38034660 con 0x7fae4815a370 2023-10-19T10:52:44.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 -- 172.21.15.50:0/495862485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 msgr2=0x7fae4806fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 --2- 172.21.15.50:0/495862485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae4806fc80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d90 tx=0x7fae3802beb0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 -- 172.21.15.50:0/495862485 shutdown_connections 2023-10-19T10:52:44.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 --2- 172.21.15.50:0/495862485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae4806fc80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 -- 172.21.15.50:0/495862485 >> 172.21.15.50:0/495862485 conn(0x7fae480ccf30 msgr2=0x7fae480cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:44.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 -- 172.21.15.50:0/495862485 shutdown_connections 2023-10-19T10:52:44.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.902+0000 7fae4d1de640 1 -- 172.21.15.50:0/495862485 wait complete. 2023-10-19T10:52:44.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 Processor -- start 2023-10-19T10:52:44.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 -- start start 2023-10-19T10:52:44.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae480b1f30 0x7fae4815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae480b23c0 0x7fae480afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae481460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4815b070 con 0x7fae4815a370 2023-10-19T10:52:44.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48146f80 con 0x7fae480b1f30 2023-10-19T10:52:44.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.904+0000 7fae4d1de640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4815c4d0 con 0x7fae480b23c0 2023-10-19T10:52:44.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae481460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae481460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56250/0 (socket says 172.21.15.50:56250) 2023-10-19T10:52:44.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 -- 172.21.15.50:0/2500487712 learned_addr learned my addr 172.21.15.50:0/2500487712 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:44.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4659c640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae480b23c0 0x7fae480afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae46d9d640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae480b1f30 0x7fae4815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 -- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae480b23c0 msgr2=0x7fae480afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae480b23c0 0x7fae480afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 -- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae480b1f30 msgr2=0x7fae4815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:44.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae480b1f30 0x7fae4815d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:44.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 -- 172.21.15.50:0/2500487712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae38009a70 con 0x7fae4815a370 2023-10-19T10:52:44.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4759e640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae481460c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fae4000b410 tx=0x7fae4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40004020 con 0x7fae4815a370 2023-10-19T10:52:44.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40004180 con 0x7fae4815a370 2023-10-19T10:52:44.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4815ad90 con 0x7fae4815a370 2023-10-19T10:52:44.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40011630 con 0x7fae4815a370 2023-10-19T10:52:44.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.905+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4815ca50 con 0x7fae4815a370 2023-10-19T10:52:44.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.906+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae48073990 con 0x7fae4815a370 2023-10-19T10:52:44.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.907+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae40011790 con 0x7fae4815a370 2023-10-19T10:52:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.907+0000 7fae3f7fe640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fae2806a050 0x7fae2806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.907+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fae400882f0 con 0x7fae4815a370 2023-10-19T10:52:44.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.907+0000 7fae46d9d640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fae2806a050 0x7fae2806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:44.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.908+0000 7fae46d9d640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fae2806a050 0x7fae2806c510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fae38036000 tx=0x7fae38030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:44.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:44.911+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fae400538e0 con 0x7fae4815a370 2023-10-19T10:52:45.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.089+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae480b2e10 con 0x7fae4815a370 2023-10-19T10:52:45.093 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.091+0000 7fae3f7fe640 1 -- 172.21.15.50:0/2500487712 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18701 (secure 0 0 0) 0x7fae40056f00 con 0x7fae4815a370 2023-10-19T10:52:45.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:45.094 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:46:26.319171+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:45.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fae2806a050 msgr2=0x7fae2806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fae2806a050 0x7fae2806c510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fae38036000 tx=0x7fae38030000 comp rx=0 tx=0).stop 2023-10-19T10:52:45.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 msgr2=0x7fae481460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae481460c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fae4000b410 tx=0x7fae4000b8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 shutdown_connections 2023-10-19T10:52:45.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fae2806a050 0x7fae2806c510 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae4815a370 0x7fae481460c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae480b23c0 0x7fae480afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 --2- 172.21.15.50:0/2500487712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae480b1f30 0x7fae4815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 >> 172.21.15.50:0/2500487712 conn(0x7fae480ccf30 msgr2=0x7fae480cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:45.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 shutdown_connections 2023-10-19T10:52:45.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.093+0000 7fae4d1de640 1 -- 172.21.15.50:0/2500487712 wait complete. 2023-10-19T10:52:45.107 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-19T10:52:45.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:52:45.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.211+0000 7fd54e2b6640 1 Processor -- start 2023-10-19T10:52:45.214 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.211+0000 7fd54e2b6640 1 -- start start 2023-10-19T10:52:45.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd54e2b6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd54815c570 0x7fd5480d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd54e2b6640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54806aa30 con 0x7fd54815c950 2023-10-19T10:52:45.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd54e2b6640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54806b2d0 con 0x7fd5480cd920 2023-10-19T10:52:45.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd54e2b6640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5480ce090 con 0x7fd54815c570 2023-10-19T10:52:45.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd5477fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd54815c950 0x7fd54806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50112/0 (socket says 172.21.15.50:50112) 2023-10-19T10:52:45.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd5477fe640 1 -- 172.21.15.50:0/1402808586 learned_addr learned my addr 172.21.15.50:0/1402808586 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:45.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd547fff640 1 --2- 172.21.15.50:0/1402808586 >> v2:172.21.15.183:3300/0 conn(0x7fd54815c570 0x7fd5480d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd546ffd640 1 -- 172.21.15.50:0/1402808586 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588101540 0 0) 0x7fd54806aa30 con 0x7fd54815c950 2023-10-19T10:52:45.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd546ffd640 1 -- 172.21.15.50:0/1402808586 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd530003520 con 0x7fd54815c950 2023-10-19T10:52:45.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd547fff640 1 -- 172.21.15.50:0/1402808586 >> v1:172.21.15.183:6789/0 conn(0x7fd5480cd920 legacy=0x7fd5480d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd547fff640 1 -- 172.21.15.50:0/1402808586 >> v1:172.21.15.50:6789/0 conn(0x7fd54815c950 legacy=0x7fd54806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.212+0000 7fd547fff640 1 -- 172.21.15.50:0/1402808586 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54806ac80 con 0x7fd54815c570 2023-10-19T10:52:45.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.213+0000 7fd547fff640 1 --2- 172.21.15.50:0/1402808586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c570 0x7fd5480d0e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009790 tx=0x7fd53c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546fef25afbcf04 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.213+0000 7fd546ffd640 1 -- 172.21.15.50:0/1402808586 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c02d040 con 0x7fd54815c570 2023-10-19T10:52:45.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.213+0000 7fd546ffd640 1 -- 172.21.15.50:0/1402808586 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c031810 con 0x7fd54815c570 2023-10-19T10:52:45.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd546ffd640 1 -- 172.21.15.50:0/1402808586 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c031b80 con 0x7fd54815c570 2023-10-19T10:52:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 -- 172.21.15.50:0/1402808586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c570 msgr2=0x7fd5480d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/1402808586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c570 0x7fd5480d0e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009790 tx=0x7fd53c02b8b0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 -- 172.21.15.50:0/1402808586 shutdown_connections 2023-10-19T10:52:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/1402808586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c570 0x7fd5480d0e30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 -- 172.21.15.50:0/1402808586 >> 172.21.15.50:0/1402808586 conn(0x7fd548089ff0 msgr2=0x7fd54809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:45.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 -- 172.21.15.50:0/1402808586 shutdown_connections 2023-10-19T10:52:45.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.214+0000 7fd54e2b6640 1 -- 172.21.15.50:0/1402808586 wait complete. 2023-10-19T10:52:45.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.215+0000 7fd54e2b6640 1 Processor -- start 2023-10-19T10:52:45.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.215+0000 7fd54e2b6640 1 -- start start 2023-10-19T10:52:45.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54e2b6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 0x7fd5480c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54e2b6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd54815c570 0x7fd54815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54e2b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c950 0x7fd54815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54e2b6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548159100 con 0x7fd5480cd920 2023-10-19T10:52:45.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54e2b6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480d3230 con 0x7fd54815c950 2023-10-19T10:52:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54e2b6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c8c30 con 0x7fd54815c570 2023-10-19T10:52:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd547fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 0x7fd5480c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd547fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 0x7fd5480c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56262/0 (socket says 172.21.15.50:56262) 2023-10-19T10:52:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd547fff640 1 -- 172.21.15.50:0/147869390 learned_addr learned my addr 172.21.15.50:0/147869390 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:45.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd5477fe640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd54815c570 0x7fd54815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.216+0000 7fd54c853640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c950 0x7fd54815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd547fff640 1 -- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd54815c570 msgr2=0x7fd54815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd547fff640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd54815c570 0x7fd54815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd547fff640 1 -- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c950 msgr2=0x7fd54815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd547fff640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c950 0x7fd54815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd547fff640 1 -- 172.21.15.50:0/147869390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c0093a0 con 0x7fd5480cd920 2023-10-19T10:52:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd547fff640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 0x7fd5480c6c50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0098b0 tx=0x7fd53c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c005ba0 con 0x7fd5480cd920 2023-10-19T10:52:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c005d00 con 0x7fd5480cd920 2023-10-19T10:52:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c003850 con 0x7fd5480cd920 2023-10-19T10:52:45.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5480c8f10 con 0x7fd5480cd920 2023-10-19T10:52:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.217+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54815e4e0 con 0x7fd5480cd920 2023-10-19T10:52:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.218+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54815e6d0 con 0x7fd5480cd920 2023-10-19T10:52:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.218+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd53c0039b0 con 0x7fd5480cd920 2023-10-19T10:52:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.219+0000 7fd544ff9640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd53406a080 0x7fd53406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.219+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd53c041030 con 0x7fd5480cd920 2023-10-19T10:52:45.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.219+0000 7fd5477fe640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd53406a080 0x7fd53406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.220+0000 7fd5477fe640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd53406a080 0x7fd53406c540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd5480d2a20 tx=0x7fd538009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.223+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd53c076030 con 0x7fd5480cd920 2023-10-19T10:52:45.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.436+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd548073990 con 0x7fd5480cd920 2023-10-19T10:52:45.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.437+0000 7fd544ff9640 1 -- 172.21.15.50:0/147869390 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fd53c035090 con 0x7fd5480cd920 2023-10-19T10:52:45.440 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:45.440 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:52:45.441 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:52:45.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd53406a080 msgr2=0x7fd53406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd53406a080 0x7fd53406c540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd5480d2a20 tx=0x7fd538009040 comp rx=0 tx=0).stop 2023-10-19T10:52:45.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 msgr2=0x7fd5480c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 0x7fd5480c6c50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0098b0 tx=0x7fd53c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 shutdown_connections 2023-10-19T10:52:45.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd53406a080 0x7fd53406c540 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd54815c950 0x7fd54815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd54815c570 0x7fd54815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 --2- 172.21.15.50:0/147869390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5480cd920 0x7fd5480c6c50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.438+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 >> 172.21.15.50:0/147869390 conn(0x7fd548089ff0 msgr2=0x7fd5480af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:45.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.439+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 shutdown_connections 2023-10-19T10:52:45.444 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.439+0000 7fd54e2b6640 1 -- 172.21.15.50:0/147869390 wait complete. 2023-10-19T10:52:45.452 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:52:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.556+0000 7fc561021640 1 Processor -- start 2023-10-19T10:52:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.556+0000 7fc561021640 1 -- start start 2023-10-19T10:52:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc561021640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc55c15c570 0x7fc55c0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc561021640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c0cd920 0x7fc55c0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc561021640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c06aa30 con 0x7fc55c15c950 2023-10-19T10:52:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc561021640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c06b2d0 con 0x7fc55c15c570 2023-10-19T10:52:45.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc561021640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0ce090 con 0x7fc55c0cd920 2023-10-19T10:52:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55a59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc55c15c950 0x7fc55c06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50116/0 (socket says 172.21.15.50:50116) 2023-10-19T10:52:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55a59c640 1 -- 172.21.15.50:0/1747600198 learned_addr learned my addr 172.21.15.50:0/1747600198 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55ad9d640 1 --2- 172.21.15.50:0/1747600198 >> v2:172.21.15.183:3300/0 conn(0x7fc55c15c570 0x7fc55c0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55b59e640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c0cd920 0x7fc55c0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc559d9b640 1 -- 172.21.15.50:0/1747600198 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065738222 0 0) 0x7fc55c06aa30 con 0x7fc55c15c950 2023-10-19T10:52:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc559d9b640 1 -- 172.21.15.50:0/1747600198 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc548003520 con 0x7fc55c15c950 2023-10-19T10:52:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc559d9b640 1 -- 172.21.15.50:0/1747600198 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422595587 0 0) 0x7fc548003520 con 0x7fc55c15c950 2023-10-19T10:52:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55ad9d640 1 -- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c0cd920 msgr2=0x7fc55c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:45.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55ad9d640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c0cd920 0x7fc55c0d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55ad9d640 1 -- 172.21.15.50:0/1747600198 >> v1:172.21.15.50:6789/0 conn(0x7fc55c15c950 legacy=0x7fc55c06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55ad9d640 1 -- 172.21.15.50:0/1747600198 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c06ac80 con 0x7fc55c15c570 2023-10-19T10:52:45.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.557+0000 7fc55b59e640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c0cd920 0x7fc55c0d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:45.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.558+0000 7fc55ad9d640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c15c570 0x7fc55c0d0e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc554009790 tx=0x7fc55402b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab017819aee48078 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.558+0000 7fc559d9b640 1 -- 172.21.15.50:0/1747600198 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544001670 con 0x7fc55c15c570 2023-10-19T10:52:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.558+0000 7fc559d9b640 1 -- 172.21.15.50:0/1747600198 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55402e050 con 0x7fc55c15c570 2023-10-19T10:52:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.558+0000 7fc559d9b640 1 -- 172.21.15.50:0/1747600198 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554030710 con 0x7fc55c15c570 2023-10-19T10:52:45.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 -- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c15c570 msgr2=0x7fc55c0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c15c570 0x7fc55c0d0e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc554009790 tx=0x7fc55402b6e0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 -- 172.21.15.50:0/1747600198 shutdown_connections 2023-10-19T10:52:45.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c0cd920 0x7fc55c0d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 --2- 172.21.15.50:0/1747600198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c15c570 0x7fc55c0d0e30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 -- 172.21.15.50:0/1747600198 >> 172.21.15.50:0/1747600198 conn(0x7fc55c089ff0 msgr2=0x7fc55c09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:45.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 -- 172.21.15.50:0/1747600198 shutdown_connections 2023-10-19T10:52:45.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.559+0000 7fc561021640 1 -- 172.21.15.50:0/1747600198 wait complete. 2023-10-19T10:52:45.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.560+0000 7fc561021640 1 Processor -- start 2023-10-19T10:52:45.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.560+0000 7fc561021640 1 -- start start 2023-10-19T10:52:45.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc561021640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c0cd920 0x7fc55c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc561021640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 0x7fc55c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc561021640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c15c950 0x7fc55c0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc561021640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c15e480 con 0x7fc55c15c570 2023-10-19T10:52:45.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc561021640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c91c0 con 0x7fc55c0cd920 2023-10-19T10:52:45.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc561021640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c70f0 con 0x7fc55c15c950 2023-10-19T10:52:45.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc55a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 0x7fc55c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc55a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 0x7fc55c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56278/0 (socket says 172.21.15.50:56278) 2023-10-19T10:52:45.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc55a59c640 1 -- 172.21.15.50:0/357949487 learned_addr learned my addr 172.21.15.50:0/357949487 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:45.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc55ad9d640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c0cd920 0x7fc55c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.561+0000 7fc55b59e640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c15c950 0x7fc55c0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc55a59c640 1 -- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c15c950 msgr2=0x7fc55c0c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc55a59c640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c15c950 0x7fc55c0c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc55a59c640 1 -- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c0cd920 msgr2=0x7fc55c159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc55a59c640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c0cd920 0x7fc55c159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc55a59c640 1 -- 172.21.15.50:0/357949487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009a70 con 0x7fc55c15c570 2023-10-19T10:52:45.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc55a59c640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 0x7fc55c0c8740 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc54400c780 tx=0x7fc54400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544005ad0 con 0x7fc55c15c570 2023-10-19T10:52:45.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc544005c50 con 0x7fc55c15c570 2023-10-19T10:52:45.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5540093a0 con 0x7fc55c15c570 2023-10-19T10:52:45.580 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544007070 con 0x7fc55c15c570 2023-10-19T10:52:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.562+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c15de30 con 0x7fc55c15c570 2023-10-19T10:52:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.563+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c15e630 con 0x7fc55c15c570 2023-10-19T10:52:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.563+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc5440071d0 con 0x7fc55c15c570 2023-10-19T10:52:45.581 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.564+0000 7fc5537fe640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc53c06a080 0x7fc53c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.564+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fc544088440 con 0x7fc55c15c570 2023-10-19T10:52:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.564+0000 7fc55ad9d640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc53c06a080 0x7fc53c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.582 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.565+0000 7fc55ad9d640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc53c06a080 0x7fc53c06c540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc554034000 tx=0x7fc554008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.583 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.568+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc55c15e630 con 0x7fc55c15c570 2023-10-19T10:52:45.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.778+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc55c15e630 con 0x7fc55c15c570 2023-10-19T10:52:45.782 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.779+0000 7fc5537fe640 1 -- 172.21.15.50:0/357949487 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fc55c15e630 con 0x7fc55c15c570 2023-10-19T10:52:45.783 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:45.783 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-19T10:38:27.942428+0000","modified":"2023-10-19T10:46:29.334941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":123,"addr":"172.21.15.50:6834/1814941523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1814941523},{"type":"v1","addr":"172.21.15.50:6834","nonce":1814941523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-19T10:52:45.785 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2023-10-19T10:52:45.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc53c06a080 msgr2=0x7fc53c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc53c06a080 0x7fc53c06c540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc554034000 tx=0x7fc554008030 comp rx=0 tx=0).stop 2023-10-19T10:52:45.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 msgr2=0x7fc55c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 0x7fc55c0c8740 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc54400c780 tx=0x7fc54400cc40 comp rx=0 tx=0).stop 2023-10-19T10:52:45.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 shutdown_connections 2023-10-19T10:52:45.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc53c06a080 0x7fc53c06c540 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc55c15c950 0x7fc55c0c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55c15c570 0x7fc55c0c8740 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 --2- 172.21.15.50:0/357949487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc55c0cd920 0x7fc55c159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 >> 172.21.15.50:0/357949487 conn(0x7fc55c089ff0 msgr2=0x7fc55c0b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:45.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 shutdown_connections 2023-10-19T10:52:45.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.781+0000 7fc561021640 1 -- 172.21.15.50:0/357949487 wait complete. 2023-10-19T10:52:45.795 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.898+0000 7f3fb95d9640 1 Processor -- start 2023-10-19T10:52:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.898+0000 7f3fb95d9640 1 -- start start 2023-10-19T10:52:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb95d9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fb415cd90 0x7f3fb406b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb95d9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb40ca480 con 0x7f3fb4159280 2023-10-19T10:52:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb95d9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb4159990 con 0x7f3fb40cd7d0 2023-10-19T10:52:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb95d9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb415d500 con 0x7f3fb415cd90 2023-10-19T10:52:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fb415cd90 0x7f3fb406b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb2ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3fb4159280 0x7f3fb40c8660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50120/0 (socket says 172.21.15.50:50120) 2023-10-19T10:52:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fb415cd90 0x7f3fb406b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56282/0 (socket says 172.21.15.50:56282) 2023-10-19T10:52:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb2ffd640 1 -- 172.21.15.50:0/988927289 learned_addr learned my addr 172.21.15.50:0/988927289 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb37fe640 1 -- 172.21.15.50:0/988927289 >> v1:172.21.15.183:6789/0 conn(0x7f3fb40cd7d0 legacy=0x7f3fb4159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb37fe640 1 -- 172.21.15.50:0/988927289 >> v1:172.21.15.50:6789/0 conn(0x7f3fb4159280 legacy=0x7f3fb40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb1ffb640 1 -- 172.21.15.50:0/988927289 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481189806 0 0) 0x7f3fb40ca480 con 0x7f3fb4159280 2023-10-19T10:52:45.905 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.899+0000 7f3fb37fe640 1 -- 172.21.15.50:0/988927289 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb415d7e0 con 0x7f3fb415cd90 2023-10-19T10:52:45.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.900+0000 7f3fb37fe640 1 --2- 172.21.15.50:0/988927289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb406b4a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009d90 tx=0x7f3fa402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71046125e92d3e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.900+0000 7f3fb1ffb640 1 -- 172.21.15.50:0/988927289 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa402cc60 con 0x7f3fb415cd90 2023-10-19T10:52:45.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.900+0000 7f3fb1ffb640 1 -- 172.21.15.50:0/988927289 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa402cdc0 con 0x7f3fb415cd90 2023-10-19T10:52:45.906 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.900+0000 7f3fb1ffb640 1 -- 172.21.15.50:0/988927289 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4034660 con 0x7f3fb415cd90 2023-10-19T10:52:45.909 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 -- 172.21.15.50:0/988927289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 msgr2=0x7f3fb406b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/988927289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb406b4a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009d90 tx=0x7f3fa402beb0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 -- 172.21.15.50:0/988927289 shutdown_connections 2023-10-19T10:52:45.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/988927289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb406b4a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 -- 172.21.15.50:0/988927289 >> 172.21.15.50:0/988927289 conn(0x7f3fb4069e90 msgr2=0x7f3fb40afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:45.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 -- 172.21.15.50:0/988927289 shutdown_connections 2023-10-19T10:52:45.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.901+0000 7f3fb95d9640 1 -- 172.21.15.50:0/988927289 wait complete. 2023-10-19T10:52:45.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.902+0000 7f3fb95d9640 1 Processor -- start 2023-10-19T10:52:45.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.902+0000 7f3fb95d9640 1 -- start start 2023-10-19T10:52:45.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb95d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fb40cd7d0 0x7f3fb4156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb95d9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fb4159280 0x7f3fb4157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb95d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb41eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb95d9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb41584c0 con 0x7f3fb415cd90 2023-10-19T10:52:45.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb95d9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb41578b0 con 0x7f3fb40cd7d0 2023-10-19T10:52:45.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb95d9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb4157b90 con 0x7f3fb4159280 2023-10-19T10:52:45.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb41eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb41eeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56298/0 (socket says 172.21.15.50:56298) 2023-10-19T10:52:45.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 -- 172.21.15.50:0/545098421 learned_addr learned my addr 172.21.15.50:0/545098421 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:45.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb2ffd640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fb40cd7d0 0x7f3fb4156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb27fc640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fb4159280 0x7f3fb4157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 -- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fb4159280 msgr2=0x7f3fb4157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fb4159280 0x7f3fb4157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 -- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fb40cd7d0 msgr2=0x7f3fb4156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:45.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fb40cd7d0 0x7f3fb4156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:45.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.903+0000 7f3fb37fe640 1 -- 172.21.15.50:0/545098421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa4009a70 con 0x7f3fb415cd90 2023-10-19T10:52:45.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.904+0000 7f3fb37fe640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb41eeb80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009d90 tx=0x7f3fa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.904+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4040450 con 0x7f3fb415cd90 2023-10-19T10:52:45.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.904+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa40405d0 con 0x7f3fb415cd90 2023-10-19T10:52:45.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.904+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb4157e70 con 0x7f3fb415cd90 2023-10-19T10:52:45.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.904+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4040980 con 0x7f3fb415cd90 2023-10-19T10:52:45.919 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.904+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb41575b0 con 0x7f3fb415cd90 2023-10-19T10:52:45.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.905+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb40c19b0 con 0x7f3fb415cd90 2023-10-19T10:52:45.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.905+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3fa4040ae0 con 0x7f3fb415cd90 2023-10-19T10:52:45.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.906+0000 7f3fab7fe640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f9406a050 0x7f3f9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:45.920 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.906+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3fa40ab360 con 0x7f3fb415cd90 2023-10-19T10:52:45.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.906+0000 7f3fb2ffd640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f9406a050 0x7f3f9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:45.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.906+0000 7f3fb2ffd640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f9406a050 0x7f3f9406c510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3fac000f60 tx=0x7f3fac007b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:45.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:45.910+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f3fa4076a00 con 0x7f3fb415cd90 2023-10-19T10:52:46.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.088+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fb415a310 con 0x7f3fb415cd90 2023-10-19T10:52:46.092 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.089+0000 7f3fab7fe640 1 -- 172.21.15.50:0/545098421 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18701 (secure 0 0 0) 0x7f3fb415a310 con 0x7f3fb415cd90 2023-10-19T10:52:46.092 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:46.092 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":55,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:46:26.319171+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-19T10:38:26.398890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-19T10:38:26.910608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:46.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f9406a050 msgr2=0x7f3f9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f9406a050 0x7f3f9406c510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3fac000f60 tx=0x7f3fac007b90 comp rx=0 tx=0).stop 2023-10-19T10:52:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 msgr2=0x7f3fb41eeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb41eeb80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009d90 tx=0x7f3fa4004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 shutdown_connections 2023-10-19T10:52:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f3f9406a050 0x7f3f9406c510 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fb415cd90 0x7f3fb41eeb80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fb4159280 0x7f3fb4157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 --2- 172.21.15.50:0/545098421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fb40cd7d0 0x7f3fb4156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 >> 172.21.15.50:0/545098421 conn(0x7f3fb4069e90 msgr2=0x7f3fb4071470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 shutdown_connections 2023-10-19T10:52:46.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.091+0000 7f3fb95d9640 1 -- 172.21.15.50:0/545098421 wait complete. 2023-10-19T10:52:46.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-19T10:52:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.210+0000 7fd398f4e640 1 Processor -- start 2023-10-19T10:52:46.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.210+0000 7fd398f4e640 1 -- start start 2023-10-19T10:52:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd398f4e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940d0e30 0x7fd39415c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd398f4e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39406aa30 con 0x7fd39415c570 2023-10-19T10:52:46.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd398f4e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39406b2d0 con 0x7fd3940cd920 2023-10-19T10:52:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd398f4e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39415ce90 con 0x7fd3940d0e30 2023-10-19T10:52:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd39259c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd39415c570 0x7fd39406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50134/0 (socket says 172.21.15.50:50134) 2023-10-19T10:52:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd39259c640 1 -- 172.21.15.50:0/2956027904 learned_addr learned my addr 172.21.15.50:0/2956027904 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd389d9b640 1 --2- 172.21.15.50:0/2956027904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940d0e30 0x7fd39415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264033800 0 0) 0x7fd39406aa30 con 0x7fd39415c570 2023-10-19T10:52:46.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd380003520 con 0x7fd39415c570 2023-10-19T10:52:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852796334 0 0) 0x7fd39406b2d0 con 0x7fd3940cd920 2023-10-19T10:52:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd39406aa30 con 0x7fd3940cd920 2023-10-19T10:52:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 571268146 0 0) 0x7fd380003520 con 0x7fd39415c570 2023-10-19T10:52:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd389d9b640 1 -- 172.21.15.50:0/2956027904 >> v1:172.21.15.183:6789/0 conn(0x7fd3940cd920 legacy=0x7fd3940d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd389d9b640 1 -- 172.21.15.50:0/2956027904 >> v1:172.21.15.50:6789/0 conn(0x7fd39415c570 legacy=0x7fd39406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.211+0000 7fd389d9b640 1 -- 172.21.15.50:0/2956027904 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940d1270 con 0x7fd3940d0e30 2023-10-19T10:52:46.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.212+0000 7fd389d9b640 1 --2- 172.21.15.50:0/2956027904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940d0e30 0x7fd39415c950 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097d0 tx=0x7fd37c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7749a1592c61de server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:46.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.212+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840017f0 con 0x7fd3940d0e30 2023-10-19T10:52:46.222 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.212+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c02e050 con 0x7fd3940d0e30 2023-10-19T10:52:46.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.212+0000 7fd391d9b640 1 -- 172.21.15.50:0/2956027904 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c030730 con 0x7fd3940d0e30 2023-10-19T10:52:46.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 -- 172.21.15.50:0/2956027904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940d0e30 msgr2=0x7fd39415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 --2- 172.21.15.50:0/2956027904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940d0e30 0x7fd39415c950 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097d0 tx=0x7fd37c02b770 comp rx=0 tx=0).stop 2023-10-19T10:52:46.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 -- 172.21.15.50:0/2956027904 shutdown_connections 2023-10-19T10:52:46.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 --2- 172.21.15.50:0/2956027904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940d0e30 0x7fd39415c950 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.223 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 -- 172.21.15.50:0/2956027904 >> 172.21.15.50:0/2956027904 conn(0x7fd394089ff0 msgr2=0x7fd39409b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:46.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 -- 172.21.15.50:0/2956027904 shutdown_connections 2023-10-19T10:52:46.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.213+0000 7fd398f4e640 1 -- 172.21.15.50:0/2956027904 wait complete. 2023-10-19T10:52:46.224 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.214+0000 7fd398f4e640 1 Processor -- start 2023-10-19T10:52:46.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 -- start start 2023-10-19T10:52:46.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940cd920 0x7fd3940c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 0x7fd394158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd39415c570 0x7fd39415f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.225 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394158670 con 0x7fd3940d0e30 2023-10-19T10:52:46.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394159ba0 con 0x7fd39415c570 2023-10-19T10:52:46.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd398f4e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39415a630 con 0x7fd3940cd920 2023-10-19T10:52:46.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd389d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 0x7fd394158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd389d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 0x7fd394158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56308/0 (socket says 172.21.15.50:56308) 2023-10-19T10:52:46.226 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.215+0000 7fd389d9b640 1 -- 172.21.15.50:0/3286061222 learned_addr learned my addr 172.21.15.50:0/3286061222 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:46.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd389d9b640 1 -- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940cd920 msgr2=0x7fd3940c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd389d9b640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940cd920 0x7fd3940c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd389d9b640 1 -- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd39415c570 msgr2=0x7fd39415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.227 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd392d9d640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd39415c570 0x7fd39415f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd389d9b640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd39415c570 0x7fd39415f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd389d9b640 1 -- 172.21.15.50:0/3286061222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009400 con 0x7fd3940d0e30 2023-10-19T10:52:46.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd389d9b640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 0x7fd394158130 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd37c000c00 tx=0x7fd37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:46.228 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c006e10 con 0x7fd3940d0e30 2023-10-19T10:52:46.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c03c620 con 0x7fd3940d0e30 2023-10-19T10:52:46.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd394159300 con 0x7fd3940d0e30 2023-10-19T10:52:46.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c03c9d0 con 0x7fd3940d0e30 2023-10-19T10:52:46.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.216+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941598c0 con 0x7fd3940d0e30 2023-10-19T10:52:46.229 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.217+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394073990 con 0x7fd3940d0e30 2023-10-19T10:52:46.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.218+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd37c039070 con 0x7fd3940d0e30 2023-10-19T10:52:46.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.218+0000 7fd38b7fe640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd37406a050 0x7fd37406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.218+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd37c0aa0f0 con 0x7fd3940d0e30 2023-10-19T10:52:46.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.218+0000 7fd39259c640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd37406a050 0x7fd37406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.230 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.219+0000 7fd39259c640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd37406a050 0x7fd37406c510 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd384001360 tx=0x7fd3840013b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:46.231 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.222+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd37c075760 con 0x7fd3940d0e30 2023-10-19T10:52:46.434 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.431+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd39415d060 con 0x7fd3940d0e30 2023-10-19T10:52:46.435 INFO:tasks.ceph.mon.a.smithi050.stderr:2023-10-19T10:52:46.432+0000 7f8abc1be640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-19T10:52:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.435+0000 7fd38b7fe640 1 -- 172.21.15.50:0/3286061222 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd37c078d80 con 0x7fd3940d0e30 2023-10-19T10:52:46.439 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-19T10:52:46.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd37406a050 msgr2=0x7fd37406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd37406a050 0x7fd37406c510 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd384001360 tx=0x7fd3840013b0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 msgr2=0x7fd394158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 0x7fd394158130 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd37c000c00 tx=0x7fd37c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 shutdown_connections 2023-10-19T10:52:46.442 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd37406a050 0x7fd37406c510 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd39415c570 0x7fd39415f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3940d0e30 0x7fd394158130 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 --2- 172.21.15.50:0/3286061222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3940cd920 0x7fd3940c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 >> 172.21.15.50:0/3286061222 conn(0x7fd394089ff0 msgr2=0x7fd3940cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:46.443 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 shutdown_connections 2023-10-19T10:52:46.445 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.438+0000 7fd398f4e640 1 -- 172.21.15.50:0/3286061222 wait complete. 2023-10-19T10:52:46.453 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-19T10:52:46.474 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2023-10-19T10:52:46.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.559+0000 7f10a7b60640 1 Processor -- start 2023-10-19T10:52:46.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.559+0000 7f10a7b60640 1 -- start start 2023-10-19T10:52:46.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a7b60640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10a015c620 0x7f10a00b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a7b60640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 0x7f10a00d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a7b60640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00cda30 con 0x7f10a00b82f0 2023-10-19T10:52:46.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a7b60640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a00ccc10 con 0x7f10a015c620 2023-10-19T10:52:46.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a7b60640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0161110 con 0x7f10a00cfb00 2023-10-19T10:52:46.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a60fd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 0x7f10a00d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a58fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10a015c620 0x7f10a00b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a60fd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 0x7f10a00d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:58256/0 (socket says 172.21.15.50:58256) 2023-10-19T10:52:46.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a50fb640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f10a00b82f0 0x7f10a00cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:32888/0 (socket says 172.21.15.50:32888) 2023-10-19T10:52:46.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a60fd640 1 -- 172.21.15.50:0/662937034 learned_addr learned my addr 172.21.15.50:0/662937034 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:46.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a58fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10a015c620 0x7f10a00b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.50:58316/0 (socket says 172.21.15.50:58316) 2023-10-19T10:52:46.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a58fc640 1 -- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 msgr2=0x7f10a00d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:46.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a58fc640 1 --2- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 0x7f10a00d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a58fc640 1 -- 172.21.15.50:0/662937034 >> v1:172.21.15.183:6789/0 conn(0x7f10a00b82f0 legacy=0x7f10a00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a58fc640 1 -- 172.21.15.50:0/662937034 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00cce20 con 0x7f10a015c620 2023-10-19T10:52:46.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a48fa640 1 -- 172.21.15.50:0/662937034 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885027902 0 0) 0x7f10a00cda30 con 0x7f10a00b82f0 2023-10-19T10:52:46.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.560+0000 7f10a60fd640 1 --2- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 0x7f10a00d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:46.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.561+0000 7f10a58fc640 1 --2- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a015c620 0x7f10a00b7db0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f109c0097a0 tx=0x7f109c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f00f65ef46601c server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:46.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.561+0000 7f10a48fa640 1 -- 172.21.15.50:0/662937034 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c02d040 con 0x7f10a015c620 2023-10-19T10:52:46.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a48fa640 1 -- 172.21.15.50:0/662937034 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c031810 con 0x7f10a015c620 2023-10-19T10:52:46.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a48fa640 1 -- 172.21.15.50:0/662937034 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c031ba0 con 0x7f10a015c620 2023-10-19T10:52:46.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 -- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a015c620 msgr2=0x7f10a00b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 --2- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a015c620 0x7f10a00b7db0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f109c0097a0 tx=0x7f109c02b8b0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 -- 172.21.15.50:0/662937034 shutdown_connections 2023-10-19T10:52:46.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 --2- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a00cfb00 0x7f10a00d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 --2- 172.21.15.50:0/662937034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a015c620 0x7f10a00b7db0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 -- 172.21.15.50:0/662937034 >> 172.21.15.50:0/662937034 conn(0x7f10a0071110 msgr2=0x7f10a00714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:46.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 -- 172.21.15.50:0/662937034 shutdown_connections 2023-10-19T10:52:46.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.562+0000 7f10a7b60640 1 -- 172.21.15.50:0/662937034 wait complete. 2023-10-19T10:52:46.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.563+0000 7f10a7b60640 1 Processor -- start 2023-10-19T10:52:46.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.563+0000 7f10a7b60640 1 -- start start 2023-10-19T10:52:46.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a7b60640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a00b82f0 0x7f10a01f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a7b60640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 0x7f10a00c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a7b60640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a015c620 0x7f10a00b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a7b60640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a015ca90 con 0x7f10a00cfb00 2023-10-19T10:52:46.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a7b60640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00c9bd0 con 0x7f10a00b82f0 2023-10-19T10:52:46.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a7b60640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b0e10 con 0x7f10a015c620 2023-10-19T10:52:46.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 0x7f10a00c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 0x7f10a00c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56348/0 (socket says 172.21.15.50:56348) 2023-10-19T10:52:46.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 -- 172.21.15.50:0/1416005747 learned_addr learned my addr 172.21.15.50:0/1416005747 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:46.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a58fc640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a00b82f0 0x7f10a01f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a60fd640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a015c620 0x7f10a00b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 -- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a015c620 msgr2=0x7f10a00b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a015c620 0x7f10a00b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 -- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a00b82f0 msgr2=0x7f10a01f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:46.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a00b82f0 0x7f10a01f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a50fb640 1 -- 172.21.15.50:0/1416005747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098009b30 con 0x7f10a00cfb00 2023-10-19T10:52:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a58fc640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a00b82f0 0x7f10a01f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.564+0000 7f10a60fd640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a015c620 0x7f10a00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.565+0000 7f10a50fb640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 0x7f10a00c9690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f109000c4b0 tx=0x7f109000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.565+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900074e0 con 0x7f10a00cfb00 2023-10-19T10:52:46.576 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.565+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090007660 con 0x7f10a00cfb00 2023-10-19T10:52:46.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.565+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c009400 con 0x7f10a00cfb00 2023-10-19T10:52:46.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.565+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090004a20 con 0x7f10a00cfb00 2023-10-19T10:52:46.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.565+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a00ca110 con 0x7f10a00cfb00 2023-10-19T10:52:46.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.566+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0073990 con 0x7f10a00cfb00 2023-10-19T10:52:46.577 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.566+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1090004b80 con 0x7f10a00cfb00 2023-10-19T10:52:46.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.567+0000 7f10967fc640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f108406a080 0x7f108406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:46.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.567+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f1090087930 con 0x7f10a00cfb00 2023-10-19T10:52:46.578 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.567+0000 7f10a58fc640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f108406a080 0x7f108406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:46.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.567+0000 7f10a58fc640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f108406a080 0x7f108406c540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f109c036000 tx=0x7f109c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:46.579 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.571+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1090052ee0 con 0x7f10a00cfb00 2023-10-19T10:52:46.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:46.784+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f10a006b6f0 con 0x7f10a00cfb00 2023-10-19T10:52:47.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.465+0000 7f10967fc640 1 -- 172.21.15.50:0/1416005747 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1090056500 con 0x7f10a00cfb00 2023-10-19T10:52:47.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f108406a080 msgr2=0x7f108406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:47.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f108406a080 0x7f108406c540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f109c036000 tx=0x7f109c03c040 comp rx=0 tx=0).stop 2023-10-19T10:52:47.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 msgr2=0x7f10a00c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:47.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 0x7f10a00c9690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f109000c4b0 tx=0x7f109000c970 comp rx=0 tx=0).stop 2023-10-19T10:52:47.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 shutdown_connections 2023-10-19T10:52:47.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f108406a080 0x7f108406c540 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a015c620 0x7f10a00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10a00cfb00 0x7f10a00c9690 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 --2- 172.21.15.50:0/1416005747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a00b82f0 0x7f10a01f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 >> 172.21.15.50:0/1416005747 conn(0x7f10a0071110 msgr2=0x7f10a00b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:47.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 shutdown_connections 2023-10-19T10:52:47.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.468+0000 7f10a7b60640 1 -- 172.21.15.50:0/1416005747 wait complete. 2023-10-19T10:52:47.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-19T10:52:47.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.637+0000 7fb982058640 1 Processor -- start 2023-10-19T10:52:47.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.637+0000 7fb982058640 1 -- start start 2023-10-19T10:52:47.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb982058640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb97c15c620 0x7fb97c0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:47.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb982058640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c0cfb00 0x7fb97c0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:47.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb982058640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0cda30 con 0x7fb97c0b82f0 2023-10-19T10:52:47.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb982058640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c0ccc10 con 0x7fb97c15c620 2023-10-19T10:52:47.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb982058640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c161110 con 0x7fb97c0cfb00 2023-10-19T10:52:47.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb97affd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb97c0b82f0 0x7fb97c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42624/0 (socket says 172.21.15.50:42624) 2023-10-19T10:52:47.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb97affd640 1 -- 172.21.15.50:0/3321993453 learned_addr learned my addr 172.21.15.50:0/3321993453 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:47.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb97bfff640 1 --2- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c0cfb00 0x7fb97c0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:47.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb97b7fe640 1 --2- 172.21.15.50:0/3321993453 >> v2:172.21.15.183:3300/0 conn(0x7fb97c15c620 0x7fb97c0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:47.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb97a7fc640 1 -- 172.21.15.50:0/3321993453 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907602741 0 0) 0x7fb97c0cda30 con 0x7fb97c0b82f0 2023-10-19T10:52:47.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.638+0000 7fb97a7fc640 1 -- 172.21.15.50:0/3321993453 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb968003520 con 0x7fb97c0b82f0 2023-10-19T10:52:47.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.639+0000 7fb97a7fc640 1 -- 172.21.15.50:0/3321993453 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3514689146 0 0) 0x7fb968003520 con 0x7fb97c0b82f0 2023-10-19T10:52:47.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.639+0000 7fb97b7fe640 1 -- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c0cfb00 msgr2=0x7fb97c0d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:47.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.639+0000 7fb97b7fe640 1 --2- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c0cfb00 0x7fb97c0d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.639+0000 7fb97b7fe640 1 -- 172.21.15.50:0/3321993453 >> v1:172.21.15.50:6789/0 conn(0x7fb97c0b82f0 legacy=0x7fb97c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:47.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.639+0000 7fb97b7fe640 1 -- 172.21.15.50:0/3321993453 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0cce20 con 0x7fb97c15c620 2023-10-19T10:52:47.646 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.639+0000 7fb97b7fe640 1 --2- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c15c620 0x7fb97c0b7db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009d90 tx=0x7fb96c02bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab201752e36ab39 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:47.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb97a7fc640 1 -- 172.21.15.50:0/3321993453 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964001670 con 0x7fb97c15c620 2023-10-19T10:52:47.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb97a7fc640 1 -- 172.21.15.50:0/3321993453 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c02c8a0 con 0x7fb97c15c620 2023-10-19T10:52:47.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb97a7fc640 1 -- 172.21.15.50:0/3321993453 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c02cc30 con 0x7fb97c15c620 2023-10-19T10:52:47.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 -- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c15c620 msgr2=0x7fb97c0b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:47.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 --2- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c15c620 0x7fb97c0b7db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009d90 tx=0x7fb96c02bd70 comp rx=0 tx=0).stop 2023-10-19T10:52:47.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 -- 172.21.15.50:0/3321993453 shutdown_connections 2023-10-19T10:52:47.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 --2- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c0cfb00 0x7fb97c0d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 --2- 172.21.15.50:0/3321993453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c15c620 0x7fb97c0b7db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 -- 172.21.15.50:0/3321993453 >> 172.21.15.50:0/3321993453 conn(0x7fb97c071110 msgr2=0x7fb97c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:47.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 -- 172.21.15.50:0/3321993453 shutdown_connections 2023-10-19T10:52:47.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.640+0000 7fb982058640 1 -- 172.21.15.50:0/3321993453 wait complete. 2023-10-19T10:52:47.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.641+0000 7fb982058640 1 Processor -- start 2023-10-19T10:52:47.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 -- start start 2023-10-19T10:52:47.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c0b82f0 0x7fb97c159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:47.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 0x7fb97c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:47.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c15c620 0x7fb97c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:47.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0aab10 con 0x7fb97c0cfb00 2023-10-19T10:52:47.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0a92f0 con 0x7fb97c0b82f0 2023-10-19T10:52:47.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb982058640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c15ddf0 con 0x7fb97c15c620 2023-10-19T10:52:47.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb97affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 0x7fb97c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:47.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb97affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 0x7fb97c15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38982/0 (socket says 172.21.15.50:38982) 2023-10-19T10:52:47.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.642+0000 7fb97affd640 1 -- 172.21.15.50:0/1825842225 learned_addr learned my addr 172.21.15.50:0/1825842225 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:47.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97b7fe640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c0b82f0 0x7fb97c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:47.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97bfff640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c15c620 0x7fb97c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:47.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97affd640 1 -- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c15c620 msgr2=0x7fb97c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:47.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97affd640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c15c620 0x7fb97c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97affd640 1 -- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c0b82f0 msgr2=0x7fb97c159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:47.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97affd640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c0b82f0 0x7fb97c159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:47.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97affd640 1 -- 172.21.15.50:0/1825842225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009a70 con 0x7fb97c0cfb00 2023-10-19T10:52:47.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb97affd640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 0x7fb97c15d220 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb9640011e0 tx=0x7fb96400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:47.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964005ad0 con 0x7fb97c0cfb00 2023-10-19T10:52:47.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964005c30 con 0x7fb97c0cfb00 2023-10-19T10:52:47.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c15e0d0 con 0x7fb97c0cfb00 2023-10-19T10:52:47.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964007070 con 0x7fb97c0cfb00 2023-10-19T10:52:47.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.643+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c15e690 con 0x7fb97c0cfb00 2023-10-19T10:52:47.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.644+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c04e420 con 0x7fb97c0cfb00 2023-10-19T10:52:47.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.645+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb9640071d0 con 0x7fb97c0cfb00 2023-10-19T10:52:47.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.645+0000 7fb973fff640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb95c06a080 0x7fb95c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:47.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.645+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6569+0+0 (secure 0 0 0) 0x7fb964088330 con 0x7fb97c0cfb00 2023-10-19T10:52:47.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.645+0000 7fb97b7fe640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb95c06a080 0x7fb95c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:47.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.646+0000 7fb97b7fe640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb95c06a080 0x7fb95c06c540 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036000 tx=0x7fb96c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:47.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.649+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fb9640538d0 con 0x7fb97c0cfb00 2023-10-19T10:52:47.828 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:47.825+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb97c06b6f0 con 0x7fb97c0cfb00 2023-10-19T10:52:48.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.472+0000 7fb973fff640 1 -- 172.21.15.50:0/1825842225 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb964056ef0 con 0x7fb97c0cfb00 2023-10-19T10:52:48.476 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2023-10-19T10:52:48.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb95c06a080 msgr2=0x7fb95c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:48.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb95c06a080 0x7fb95c06c540 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036000 tx=0x7fb96c030000 comp rx=0 tx=0).stop 2023-10-19T10:52:48.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 msgr2=0x7fb97c15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:48.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 0x7fb97c15d220 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb9640011e0 tx=0x7fb96400cbe0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 shutdown_connections 2023-10-19T10:52:48.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fb95c06a080 0x7fb95c06c540 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb97c15c620 0x7fb97c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c0cfb00 0x7fb97c15d220 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 --2- 172.21.15.50:0/1825842225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb97c0b82f0 0x7fb97c159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.477+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 >> 172.21.15.50:0/1825842225 conn(0x7fb97c071110 msgr2=0x7fb97c0b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:48.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.478+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 shutdown_connections 2023-10-19T10:52:48.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.478+0000 7fb982058640 1 -- 172.21.15.50:0/1825842225 wait complete. 2023-10-19T10:52:48.495 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-19T10:52:48.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.609+0000 7f5da5d35640 1 Processor -- start 2023-10-19T10:52:48.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.609+0000 7f5da5d35640 1 -- start start 2023-10-19T10:52:48.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5da5d35640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5da015a520 0x7f5da006f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:48.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5da5d35640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015ea70 0x7f5da006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:48.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5da5d35640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00cae20 con 0x7f5da006fcf0 2023-10-19T10:52:48.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5da5d35640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00b6100 con 0x7f5da015a520 2023-10-19T10:52:48.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5da5d35640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015df90 con 0x7f5da015ea70 2023-10-19T10:52:48.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5da015a520 0x7f5da006f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:48.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5da015a520 0x7f5da006f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38988/0 (socket says 172.21.15.50:38988) 2023-10-19T10:52:48.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/3997590549 learned_addr learned my addr 172.21.15.50:0/3997590549 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:48.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9effd640 1 --2- 172.21.15.50:0/3997590549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015ea70 0x7f5da006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:48.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/3997590549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015ea70 msgr2=0x7f5da006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:48.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 --2- 172.21.15.50:0/3997590549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015ea70 0x7f5da006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/3997590549 >> v1:172.21.15.183:6789/0 conn(0x7f5da006fcf0 legacy=0x7f5da00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:48.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.610+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/3997590549 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da015e270 con 0x7f5da015a520 2023-10-19T10:52:48.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.611+0000 7f5d9f7fe640 1 --2- 172.21.15.50:0/3997590549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da015a520 0x7f5da006f270 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009870 tx=0x7f5d9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8369e0252a4fca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:48.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.611+0000 7f5d9e7fc640 1 -- 172.21.15.50:0/3997590549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90031640 con 0x7f5da015a520 2023-10-19T10:52:48.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.611+0000 7f5d9e7fc640 1 -- 172.21.15.50:0/3997590549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d900317a0 con 0x7f5da015a520 2023-10-19T10:52:48.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.611+0000 7f5d9e7fc640 1 -- 172.21.15.50:0/3997590549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90031b30 con 0x7f5da015a520 2023-10-19T10:52:48.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 -- 172.21.15.50:0/3997590549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da015a520 msgr2=0x7f5da006f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:48.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 --2- 172.21.15.50:0/3997590549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da015a520 0x7f5da006f270 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009870 tx=0x7f5d9002b8f0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 -- 172.21.15.50:0/3997590549 shutdown_connections 2023-10-19T10:52:48.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 --2- 172.21.15.50:0/3997590549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015ea70 0x7f5da006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 --2- 172.21.15.50:0/3997590549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da015a520 0x7f5da006f270 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 -- 172.21.15.50:0/3997590549 >> 172.21.15.50:0/3997590549 conn(0x7f5da006dad0 msgr2=0x7f5da006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:48.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 -- 172.21.15.50:0/3997590549 shutdown_connections 2023-10-19T10:52:48.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.612+0000 7f5da5d35640 1 -- 172.21.15.50:0/3997590549 wait complete. 2023-10-19T10:52:48.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.613+0000 7f5da5d35640 1 Processor -- start 2023-10-19T10:52:48.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.613+0000 7f5da5d35640 1 -- start start 2023-10-19T10:52:48.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5da5d35640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 0x7f5da014cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5da5d35640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015a520 0x7f5da00ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5da5d35640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da015ea70 0x7f5da00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5da5d35640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c9530 con 0x7f5da006fcf0 2023-10-19T10:52:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5da5d35640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c88b0 con 0x7f5da015ea70 2023-10-19T10:52:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5da5d35640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c8b90 con 0x7f5da015a520 2023-10-19T10:52:48.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 0x7f5da014cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:48.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 0x7f5da014cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38998/0 (socket says 172.21.15.50:38998) 2023-10-19T10:52:48.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/1484511661 learned_addr learned my addr 172.21.15.50:0/1484511661 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:48.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9effd640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015a520 0x7f5da00ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:48.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9ffff640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da015ea70 0x7f5da00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:48.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015a520 msgr2=0x7f5da00ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:48.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015a520 0x7f5da00ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da015ea70 msgr2=0x7f5da00b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:48.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da015ea70 0x7f5da00b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:48.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.614+0000 7f5d9f7fe640 1 -- 172.21.15.50:0/1484511661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900094d0 con 0x7f5da006fcf0 2023-10-19T10:52:48.625 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.615+0000 7f5d9f7fe640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 0x7f5da014cb20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009870 tx=0x7f5d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.615+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9003ac60 con 0x7f5da006fcf0 2023-10-19T10:52:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.615+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9003adc0 con 0x7f5da006fcf0 2023-10-19T10:52:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.615+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da00c8e70 con 0x7f5da006fcf0 2023-10-19T10:52:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.615+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90044630 con 0x7f5da006fcf0 2023-10-19T10:52:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.615+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da014d060 con 0x7f5da006fcf0 2023-10-19T10:52:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.616+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da00c9670 con 0x7f5da006fcf0 2023-10-19T10:52:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.616+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d9002f050 con 0x7f5da006fcf0 2023-10-19T10:52:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.617+0000 7f5d87fff640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d8c06a050 0x7f5d8c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.617+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6131+0+0 (secure 0 0 0) 0x7f5d900aa500 con 0x7f5da006fcf0 2023-10-19T10:52:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.617+0000 7f5d9effd640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d8c06a050 0x7f5d8c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:48.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.617+0000 7f5d9effd640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d8c06a050 0x7f5d8c06c510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006fd0 tx=0x7f5d94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:48.628 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.621+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f5da00c9670 con 0x7f5da006fcf0 2023-10-19T10:52:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:48.799+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5da00c9670 con 0x7f5da006fcf0 2023-10-19T10:52:49.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.474+0000 7f5d87fff640 1 -- 172.21.15.50:0/1484511661 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5da00c9670 con 0x7f5da006fcf0 2023-10-19T10:52:49.477 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2023-10-19T10:52:49.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d8c06a050 msgr2=0x7f5d8c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d8c06a050 0x7f5d8c06c510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006fd0 tx=0x7f5d94008040 comp rx=0 tx=0).stop 2023-10-19T10:52:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 msgr2=0x7f5da014cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 0x7f5da014cb20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009870 tx=0x7f5d900046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 shutdown_connections 2023-10-19T10:52:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5d8c06a050 0x7f5d8c06c510 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da015ea70 0x7f5da00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015a520 0x7f5da00ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 --2- 172.21.15.50:0/1484511661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5da006fcf0 0x7f5da014cb20 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.477+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 >> 172.21.15.50:0/1484511661 conn(0x7f5da006dad0 msgr2=0x7f5da00cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.478+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 shutdown_connections 2023-10-19T10:52:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.478+0000 7f5da5d35640 1 -- 172.21.15.50:0/1484511661 wait complete. 2023-10-19T10:52:49.494 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_osd_laggy_weight 2023-10-19T10:52:49.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.606+0000 7f57c16cb640 1 Processor -- start 2023-10-19T10:52:49.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.606+0000 7f57c16cb640 1 -- start start 2023-10-19T10:52:49.610 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57c16cb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f57bc0cfb00 0x7f57bc167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57c16cb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57bc0cda30 con 0x7f57bc15c620 2023-10-19T10:52:49.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57c16cb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57bc161110 con 0x7f57bc0b55c0 2023-10-19T10:52:49.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57c16cb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57bc0ccc10 con 0x7f57bc0cfb00 2023-10-19T10:52:49.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57baffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f57bc15c620 0x7f57bc0b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42632/0 (socket says 172.21.15.50:42632) 2023-10-19T10:52:49.611 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57baffd640 1 -- 172.21.15.50:0/285035422 learned_addr learned my addr 172.21.15.50:0/285035422 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:49.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57bb7fe640 1 --2- 172.21.15.50:0/285035422 >> v2:172.21.15.50:3300/0 conn(0x7f57bc0cfb00 0x7f57bc167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57bb7fe640 1 -- 172.21.15.50:0/285035422 >> v1:172.21.15.183:6789/0 conn(0x7f57bc0b55c0 legacy=0x7f57bc0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:49.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57bb7fe640 1 -- 172.21.15.50:0/285035422 >> v1:172.21.15.50:6789/0 conn(0x7f57bc15c620 legacy=0x7f57bc0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.607+0000 7f57bb7fe640 1 -- 172.21.15.50:0/285035422 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0cce20 con 0x7f57bc0cfb00 2023-10-19T10:52:49.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.608+0000 7f57bb7fe640 1 --2- 172.21.15.50:0/285035422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc0cfb00 0x7f57bc167550 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20878e4ec533711 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:49.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.608+0000 7f57ba7fc640 1 -- 172.21.15.50:0/285035422 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b402cc60 con 0x7f57bc0cfb00 2023-10-19T10:52:49.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.608+0000 7f57ba7fc640 1 -- 172.21.15.50:0/285035422 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b402cdc0 con 0x7f57bc0cfb00 2023-10-19T10:52:49.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.608+0000 7f57ba7fc640 1 -- 172.21.15.50:0/285035422 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4034660 con 0x7f57bc0cfb00 2023-10-19T10:52:49.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 -- 172.21.15.50:0/285035422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc0cfb00 msgr2=0x7f57bc167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 --2- 172.21.15.50:0/285035422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc0cfb00 0x7f57bc167550 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b402beb0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 -- 172.21.15.50:0/285035422 shutdown_connections 2023-10-19T10:52:49.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 --2- 172.21.15.50:0/285035422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc0cfb00 0x7f57bc167550 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 -- 172.21.15.50:0/285035422 >> 172.21.15.50:0/285035422 conn(0x7f57bc071110 msgr2=0x7f57bc0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:49.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 -- 172.21.15.50:0/285035422 shutdown_connections 2023-10-19T10:52:49.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.609+0000 7f57c16cb640 1 -- 172.21.15.50:0/285035422 wait complete. 2023-10-19T10:52:49.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.610+0000 7f57c16cb640 1 Processor -- start 2023-10-19T10:52:49.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.610+0000 7f57c16cb640 1 -- start start 2023-10-19T10:52:49.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57bc0b55c0 0x7f57bc0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57bc0cfb00 0x7f57bc0af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 0x7f57bc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0af7e0 con 0x7f57bc15c620 2023-10-19T10:52:49.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0afa60 con 0x7f57bc0b55c0 2023-10-19T10:52:49.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0afd40 con 0x7f57bc0cfb00 2023-10-19T10:52:49.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 0x7f57bc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 0x7f57bc14ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39008/0 (socket says 172.21.15.50:39008) 2023-10-19T10:52:49.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 -- 172.21.15.50:0/2312537878 learned_addr learned my addr 172.21.15.50:0/2312537878 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57baffd640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57bc0b55c0 0x7f57bc0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57b3fff640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57bc0cfb00 0x7f57bc0af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 -- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57bc0cfb00 msgr2=0x7f57bc0af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57bc0cfb00 0x7f57bc0af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 -- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57bc0b55c0 msgr2=0x7f57bc0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57bc0b55c0 0x7f57bc0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 -- 172.21.15.50:0/2312537878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4009a70 con 0x7f57bc15c620 2023-10-19T10:52:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57baffd640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57bc0b55c0 0x7f57bc0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57bb7fe640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 0x7f57bc14ca90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4040450 con 0x7f57bc15c620 2023-10-19T10:52:49.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b40405d0 con 0x7f57bc15c620 2023-10-19T10:52:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.611+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57bc0b0020 con 0x7f57bc15c620 2023-10-19T10:52:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.612+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4040980 con 0x7f57bc15c620 2023-10-19T10:52:49.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.612+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc0b05e0 con 0x7f57bc15c620 2023-10-19T10:52:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.613+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57bc0b07d0 con 0x7f57bc15c620 2023-10-19T10:52:49.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.613+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f57b4040ae0 con 0x7f57bc15c620 2023-10-19T10:52:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.613+0000 7f57b37fe640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f579c06a050 0x7f579c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.613+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f57b40aaf40 con 0x7f57bc15c620 2023-10-19T10:52:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.614+0000 7f57baffd640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f579c06a050 0x7f579c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.614+0000 7f57baffd640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f579c06a050 0x7f579c06c510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005880 tx=0x7f57ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:49.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.618+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f57b40769a0 con 0x7f57bc15c620 2023-10-19T10:52:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.793+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_osd_laggy_weight"} v 0) v1 -- 0x7f57bc073990 con 0x7f57bc15c620 2023-10-19T10:52:49.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.796+0000 7f57b37fe640 1 -- 172.21.15.50:0/2312537878 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_osd_laggy_weight"}]=0 v14) v1 ==== 103+0+0 (secure 0 0 0) 0x7f57b4079fc0 con 0x7f57bc15c620 2023-10-19T10:52:49.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f579c06a050 msgr2=0x7f579c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f579c06a050 0x7f579c06c510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005880 tx=0x7f57ac009040 comp rx=0 tx=0).stop 2023-10-19T10:52:49.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 msgr2=0x7f57bc14ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 0x7f57bc14ca90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b4004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:49.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 shutdown_connections 2023-10-19T10:52:49.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f579c06a050 0x7f579c06c510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57bc15c620 0x7f57bc14ca90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f57bc0cfb00 0x7f57bc0af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 --2- 172.21.15.50:0/2312537878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57bc0b55c0 0x7f57bc0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 >> 172.21.15.50:0/2312537878 conn(0x7f57bc071110 msgr2=0x7f57bc15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:49.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 shutdown_connections 2023-10-19T10:52:49.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.798+0000 7f57c16cb640 1 -- 172.21.15.50:0/2312537878 wait complete. 2023-10-19T10:52:49.812 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-19T10:52:49.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 Processor -- start 2023-10-19T10:52:49.921 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 -- start start 2023-10-19T10:52:49.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2eb015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2eb0069800 0x7f2eb0069c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb006a1a0 con 0x7f2eb00cd3c0 2023-10-19T10:52:49.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb015db10 con 0x7f2eb0069800 2023-10-19T10:52:49.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.918+0000 7f2eb4b40640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb006de30 con 0x7f2eb006d9b0 2023-10-19T10:52:49.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eae59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2eb00cd3c0 0x7f2eb00706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42636/0 (socket says 172.21.15.50:42636) 2023-10-19T10:52:49.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eae59c640 1 -- 172.21.15.50:0/1681292732 learned_addr learned my addr 172.21.15.50:0/1681292732 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:49.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2ea7fff640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2eb015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017991560 0 0) 0x7f2eb006a1a0 con 0x7f2eb00cd3c0 2023-10-19T10:52:49.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eaed9d640 1 --2- 172.21.15.50:0/1681292732 >> v2:172.21.15.183:3300/0 conn(0x7f2eb0069800 0x7f2eb0069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c003520 con 0x7f2eb00cd3c0 2023-10-19T10:52:49.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121798639 0 0) 0x7f2e9c003520 con 0x7f2eb00cd3c0 2023-10-19T10:52:49.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 msgr2=0x7f2eb015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:49.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2eb015cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 >> v2:172.21.15.183:3300/0 conn(0x7f2eb0069800 msgr2=0x7f2eb0069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:49.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 --2- 172.21.15.50:0/1681292732 >> v2:172.21.15.183:3300/0 conn(0x7f2eb0069800 0x7f2eb0069c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb00cbbf0 con 0x7f2eb00cd3c0 2023-10-19T10:52:49.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f2ea0001670 con 0x7f2eb00cd3c0 2023-10-19T10:52:49.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 >> v1:172.21.15.50:6789/0 conn(0x7f2eb00cd3c0 legacy=0x7f2eb00706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9c0039a0 0x7f2e9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e9c0047c0 0x7f2e9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2e9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb015db10 con 0x7f2e9c0039a0 2023-10-19T10:52:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb006de30 con 0x7f2e9c0047c0 2023-10-19T10:52:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c00b980 con 0x7f2eb006d9b0 2023-10-19T10:52:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eae59c640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9c0039a0 0x7f2e9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2ea7fff640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e9c0047c0 0x7f2e9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.919+0000 7f2eaed9d640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2e9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eae59c640 1 -- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 msgr2=0x7f2e9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:49.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eae59c640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2e9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eae59c640 1 -- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e9c0047c0 msgr2=0x7f2e9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:49.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eae59c640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e9c0047c0 0x7f2e9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eae59c640 1 -- 172.21.15.50:0/1681292732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c003520 con 0x7f2e9c0039a0 2023-10-19T10:52:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eaed9d640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2eb006d9b0 0x7f2e9c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.920+0000 7f2eae59c640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9c0039a0 0x7f2e9c003dc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c240 tx=0x7f2e9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e7e06ed4e0011b server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0005ad0 con 0x7f2e9c0039a0 2023-10-19T10:52:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea0005c50 con 0x7f2e9c0039a0 2023-10-19T10:52:49.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eadd9b640 1 -- 172.21.15.50:0/1681292732 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea002c8b0 con 0x7f2e9c0039a0 2023-10-19T10:52:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eb4b40640 1 -- 172.21.15.50:0/1681292732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9c0039a0 msgr2=0x7f2e9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/1681292732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9c0039a0 0x7f2e9c003dc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c240 tx=0x7f2e9c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eaed9d640 1 -- 172.21.15.50:0/1681292732 reap_dead start 2023-10-19T10:52:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eb4b40640 1 -- 172.21.15.50:0/1681292732 shutdown_connections 2023-10-19T10:52:49.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eb4b40640 1 -- 172.21.15.50:0/1681292732 >> 172.21.15.50:0/1681292732 conn(0x7f2eb00b0c60 msgr2=0x7f2eb00b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:49.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eb4b40640 1 -- 172.21.15.50:0/1681292732 shutdown_connections 2023-10-19T10:52:49.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.921+0000 7f2eb4b40640 1 -- 172.21.15.50:0/1681292732 wait complete. 2023-10-19T10:52:49.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.922+0000 7f2eb4b40640 1 Processor -- start 2023-10-19T10:52:49.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 -- start start 2023-10-19T10:52:49.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e9c0039a0 0x7f2eb00cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 0x7f2eb00c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01f1ec0 0x7f2eb01f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00c6810 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.936 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00c6200 con 0x7f2eb01f1ec0 2023-10-19T10:52:49.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2eb4b40640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb01f29d0 con 0x7f2e9c0039a0 2023-10-19T10:52:49.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2ea7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 0x7f2eb00c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2ea7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 0x7f2eb00c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39018/0 (socket says 172.21.15.50:39018) 2023-10-19T10:52:49.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.923+0000 7f2ea7fff640 1 -- 172.21.15.50:0/4055636989 learned_addr learned my addr 172.21.15.50:0/4055636989 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:49.937 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea7fff640 1 -- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e9c0039a0 msgr2=0x7f2eb00cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea7fff640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e9c0039a0 0x7f2eb00cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea7fff640 1 -- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01f1ec0 msgr2=0x7f2eb01f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea7fff640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01f1ec0 0x7f2eb01f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:49.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea7fff640 1 -- 172.21.15.50:0/4055636989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb01f2cb0 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea7fff640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 0x7f2eb00c7120 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2e980118e0 tx=0x7f2e98011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800fa70 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9800fbd0 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb00c64a0 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98014440 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.924+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb01f56e0 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.925+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb0073990 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.925+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e980145a0 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.926+0000 7f2ea6ffd640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e9006a050 0x7f2e9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.926+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2e98088cb0 con 0x7f2eb00c6ca0 2023-10-19T10:52:49.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.926+0000 7f2eae59c640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e9006a050 0x7f2e9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.927+0000 7f2eae59c640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e9006a050 0x7f2e9006c510 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0034000 tx=0x7f2ea0007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:49.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:49.930+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2e98054530 con 0x7f2eb00c6ca0 2023-10-19T10:52:50.119 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.116+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2eb00b1e60 con 0x7f2eb00c6ca0 2023-10-19T10:52:50.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.119+0000 7f2ea6ffd640 1 -- 172.21.15.50:0/4055636989 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2e98057b50 con 0x7f2eb00c6ca0 2023-10-19T10:52:50.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e9006a050 msgr2=0x7f2e9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e9006a050 0x7f2e9006c510 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0034000 tx=0x7f2ea0007000 comp rx=0 tx=0).stop 2023-10-19T10:52:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 msgr2=0x7f2eb00c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 0x7f2eb00c7120 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2e980118e0 tx=0x7f2e98011da0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 shutdown_connections 2023-10-19T10:52:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2e9006a050 0x7f2e9006c510 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01f1ec0 0x7f2eb01f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb00c6ca0 0x7f2eb00c7120 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 --2- 172.21.15.50:0/4055636989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e9c0039a0 0x7f2eb00cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 >> 172.21.15.50:0/4055636989 conn(0x7f2eb00b0c60 msgr2=0x7f2eb00cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 shutdown_connections 2023-10-19T10:52:50.127 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.121+0000 7f2eb4b40640 1 -- 172.21.15.50:0/4055636989 wait complete. 2023-10-19T10:52:50.135 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds defer_client_eviction_on_laggy_osds 2023-10-19T10:52:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.241+0000 7f05f402f640 1 Processor -- start 2023-10-19T10:52:50.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.241+0000 7f05f402f640 1 -- start start 2023-10-19T10:52:50.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.241+0000 7f05f402f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f05ec0cfb00 0x7f05ec167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.241+0000 7f05f402f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0cda30 con 0x7f05ec15c620 2023-10-19T10:52:50.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f402f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec161110 con 0x7f05ec0b55c0 2023-10-19T10:52:50.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f402f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec0ccc10 con 0x7f05ec0cfb00 2023-10-19T10:52:50.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f1dcb640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f05ec15c620 0x7f05ec0b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42640/0 (socket says 172.21.15.50:42640) 2023-10-19T10:52:50.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f1dcb640 1 -- 172.21.15.50:0/921212134 learned_addr learned my addr 172.21.15.50:0/921212134 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f25cc640 1 --2- 172.21.15.50:0/921212134 >> v2:172.21.15.183:3300/0 conn(0x7f05ec0cfb00 0x7f05ec167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680628025 0 0) 0x7f05ec0cda30 con 0x7f05ec15c620 2023-10-19T10:52:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d8003520 con 0x7f05ec15c620 2023-10-19T10:52:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537902896 0 0) 0x7f05ec161110 con 0x7f05ec0b55c0 2023-10-19T10:52:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec0cda30 con 0x7f05ec0b55c0 2023-10-19T10:52:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2026663439 0 0) 0x7f05d8003520 con 0x7f05ec15c620 2023-10-19T10:52:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 >> v2:172.21.15.183:3300/0 conn(0x7f05ec0cfb00 msgr2=0x7f05ec167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 --2- 172.21.15.50:0/921212134 >> v2:172.21.15.183:3300/0 conn(0x7f05ec0cfb00 0x7f05ec167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 >> v1:172.21.15.183:6789/0 conn(0x7f05ec0b55c0 legacy=0x7f05ec0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0cce20 con 0x7f05ec15c620 2023-10-19T10:52:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f05e8001670 con 0x7f05ec15c620 2023-10-19T10:52:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 >> v1:172.21.15.50:6789/0 conn(0x7f05ec15c620 legacy=0x7f05ec0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05d8003b00 0x7f05d8003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05d80047c0 0x7f05d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8005180 0x7f05d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0ccc10 con 0x7f05d8003b00 2023-10-19T10:52:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d800c000 con 0x7f05d8005180 2023-10-19T10:52:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d800c1c0 con 0x7f05d80047c0 2023-10-19T10:52:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.242+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05d8003b00 0x7f05d8003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f15ca640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05d80047c0 0x7f05d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f25cc640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8005180 0x7f05d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f1dcb640 1 -- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05d80047c0 msgr2=0x7f05d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05d80047c0 0x7f05d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f1dcb640 1 -- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8005180 msgr2=0x7f05d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:50.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8005180 0x7f05d800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f25cc640 1 -- 172.21.15.50:0/921212134 reap_dead start 2023-10-19T10:52:50.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f1dcb640 1 -- 172.21.15.50:0/921212134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8003520 con 0x7f05d8003b00 2023-10-19T10:52:50.254 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.243+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05d8003b00 0x7f05d8003f00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f05e800c190 tx=0x7f05d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f60293746e3b27 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:50.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8005ad0 con 0x7f05d8003b00 2023-10-19T10:52:50.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8005c50 con 0x7f05d8003b00 2023-10-19T10:52:50.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f0dc9640 1 -- 172.21.15.50:0/921212134 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e802c850 con 0x7f05d8003b00 2023-10-19T10:52:50.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 -- 172.21.15.50:0/921212134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05d8003b00 msgr2=0x7f05d8003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05d8003b00 0x7f05d8003f00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f05e800c190 tx=0x7f05d80057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 -- 172.21.15.50:0/921212134 shutdown_connections 2023-10-19T10:52:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 --2- 172.21.15.50:0/921212134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05d8003b00 0x7f05d8003f00 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 -- 172.21.15.50:0/921212134 >> 172.21.15.50:0/921212134 conn(0x7f05ec071110 msgr2=0x7f05ec0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:50.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 -- 172.21.15.50:0/921212134 shutdown_connections 2023-10-19T10:52:50.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.244+0000 7f05f402f640 1 -- 172.21.15.50:0/921212134 wait complete. 2023-10-19T10:52:50.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 Processor -- start 2023-10-19T10:52:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 -- start start 2023-10-19T10:52:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8003b00 0x7f05ec1488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05ec0cbb10 0x7f05ec14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 0x7f05ec14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec149bf0 con 0x7f05ec14bdd0 2023-10-19T10:52:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0b14e0 con 0x7f05d8003b00 2023-10-19T10:52:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f402f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec148f70 con 0x7f05ec0cbb10 2023-10-19T10:52:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f25cc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 0x7f05ec14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f25cc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 0x7f05ec14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39044/0 (socket says 172.21.15.50:39044) 2023-10-19T10:52:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f25cc640 1 -- 172.21.15.50:0/3516575652 learned_addr learned my addr 172.21.15.50:0/3516575652 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:50.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.246+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8003b00 0x7f05ec1488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f15ca640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05ec0cbb10 0x7f05ec14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f25cc640 1 -- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05ec0cbb10 msgr2=0x7f05ec14a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f25cc640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05ec0cbb10 0x7f05ec14a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f25cc640 1 -- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8003b00 msgr2=0x7f05ec1488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f25cc640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8003b00 0x7f05ec1488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f25cc640 1 -- 172.21.15.50:0/3516575652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec149250 con 0x7f05ec14bdd0 2023-10-19T10:52:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f25cc640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 0x7f05ec14c250 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f05d8004740 tx=0x7f05e400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e400e040 con 0x7f05ec14bdd0 2023-10-19T10:52:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e400f040 con 0x7f05ec14bdd0 2023-10-19T10:52:50.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec149530 con 0x7f05ec14bdd0 2023-10-19T10:52:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4003bd0 con 0x7f05ec14bdd0 2023-10-19T10:52:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.247+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec14c790 con 0x7f05ec14bdd0 2023-10-19T10:52:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.248+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec149d30 con 0x7f05ec14bdd0 2023-10-19T10:52:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.248+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f05e4003d30 con 0x7f05ec14bdd0 2023-10-19T10:52:50.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.249+0000 7f05e27fc640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f05d006a080 0x7f05d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.249+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f05e4088b10 con 0x7f05ec14bdd0 2023-10-19T10:52:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.249+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f05d006a080 0x7f05d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.250+0000 7f05f1dcb640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f05d006a080 0x7f05d006c540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05e8034000 tx=0x7f05e8007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:50.265 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.253+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f05ec149d30 con 0x7f05ec14bdd0 2023-10-19T10:52:50.433 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.430+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "defer_client_eviction_on_laggy_osds"} v 0) v1 -- 0x7f05ec149d30 con 0x7f05ec14bdd0 2023-10-19T10:52:50.436 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.433+0000 7f05e27fc640 1 -- 172.21.15.50:0/3516575652 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "defer_client_eviction_on_laggy_osds"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f05ec149d30 con 0x7f05ec14bdd0 2023-10-19T10:52:50.438 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f05d006a080 msgr2=0x7f05d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f05d006a080 0x7f05d006c540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05e8034000 tx=0x7f05e8007000 comp rx=0 tx=0).stop 2023-10-19T10:52:50.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 msgr2=0x7f05ec14c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 0x7f05ec14c250 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f05d8004740 tx=0x7f05e400f580 comp rx=0 tx=0).stop 2023-10-19T10:52:50.439 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 shutdown_connections 2023-10-19T10:52:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f05d006a080 0x7f05d006c540 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05ec14bdd0 0x7f05ec14c250 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05ec0cbb10 0x7f05ec14a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 --2- 172.21.15.50:0/3516575652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d8003b00 0x7f05ec1488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 >> 172.21.15.50:0/3516575652 conn(0x7f05ec071110 msgr2=0x7f05ec15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:50.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 shutdown_connections 2023-10-19T10:52:50.441 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.435+0000 7f05f402f640 1 -- 172.21.15.50:0/3516575652 wait complete. 2023-10-19T10:52:50.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set' 2023-10-19T10:52:50.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.554+0000 7fcfef707640 1 Processor -- start 2023-10-19T10:52:50.557 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.554+0000 7fcfef707640 1 -- start start 2023-10-19T10:52:50.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfef707640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80d0e30 0x7fcfe815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfef707640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcfe80cd920 0x7fcfe80d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfef707640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe806aa30 con 0x7fcfe815c570 2023-10-19T10:52:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfef707640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe815ce90 con 0x7fcfe80cd920 2023-10-19T10:52:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfef707640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe806b2d0 con 0x7fcfe80d0e30 2023-10-19T10:52:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfedca4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcfe80cd920 0x7fcfe80d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfecca2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80d0e30 0x7fcfe815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfed4a3640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fcfe815c570 0x7fcfe806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:40838/0 (socket says 172.21.15.50:40838) 2023-10-19T10:52:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.555+0000 7fcfed4a3640 1 -- 172.21.15.50:0/3015604943 learned_addr learned my addr 172.21.15.50:0/3015604943 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.556+0000 7fcfdffff640 1 -- 172.21.15.50:0/3015604943 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029018355 0 0) 0x7fcfe806aa30 con 0x7fcfe815c570 2023-10-19T10:52:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.556+0000 7fcfdffff640 1 -- 172.21.15.50:0/3015604943 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd4003520 con 0x7fcfe815c570 2023-10-19T10:52:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.556+0000 7fcfecca2640 1 -- 172.21.15.50:0/3015604943 >> v2:172.21.15.183:3300/0 conn(0x7fcfe80cd920 msgr2=0x7fcfe80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.556+0000 7fcfecca2640 1 --2- 172.21.15.50:0/3015604943 >> v2:172.21.15.183:3300/0 conn(0x7fcfe80cd920 0x7fcfe80d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.556+0000 7fcfecca2640 1 -- 172.21.15.50:0/3015604943 >> v1:172.21.15.183:6789/0 conn(0x7fcfe815c570 legacy=0x7fcfe806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.556+0000 7fcfecca2640 1 -- 172.21.15.50:0/3015604943 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe80d1270 con 0x7fcfe80d0e30 2023-10-19T10:52:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.557+0000 7fcfecca2640 1 --2- 172.21.15.50:0/3015604943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80d0e30 0x7fcfe815c950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80097a0 tx=0x7fcfd802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db99f230c17286e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.557+0000 7fcfdffff640 1 -- 172.21.15.50:0/3015604943 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd802e070 con 0x7fcfe80d0e30 2023-10-19T10:52:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.557+0000 7fcfdffff640 1 -- 172.21.15.50:0/3015604943 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80315a0 con 0x7fcfe80d0e30 2023-10-19T10:52:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.557+0000 7fcfdffff640 1 -- 172.21.15.50:0/3015604943 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8031930 con 0x7fcfe80d0e30 2023-10-19T10:52:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.557+0000 7fcfef707640 1 -- 172.21.15.50:0/3015604943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80d0e30 msgr2=0x7fcfe815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.557+0000 7fcfef707640 1 --2- 172.21.15.50:0/3015604943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80d0e30 0x7fcfe815c950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80097a0 tx=0x7fcfd802b840 comp rx=0 tx=0).stop 2023-10-19T10:52:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.558+0000 7fcfef707640 1 -- 172.21.15.50:0/3015604943 shutdown_connections 2023-10-19T10:52:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.558+0000 7fcfef707640 1 --2- 172.21.15.50:0/3015604943 >> v2:172.21.15.183:3300/0 conn(0x7fcfe80cd920 0x7fcfe80d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.558+0000 7fcfef707640 1 --2- 172.21.15.50:0/3015604943 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80d0e30 0x7fcfe815c950 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.558+0000 7fcfef707640 1 -- 172.21.15.50:0/3015604943 >> 172.21.15.50:0/3015604943 conn(0x7fcfe8089ff0 msgr2=0x7fcfe809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.558+0000 7fcfef707640 1 -- 172.21.15.50:0/3015604943 shutdown_connections 2023-10-19T10:52:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.558+0000 7fcfef707640 1 -- 172.21.15.50:0/3015604943 wait complete. 2023-10-19T10:52:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 Processor -- start 2023-10-19T10:52:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 -- start start 2023-10-19T10:52:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80cd920 0x7fcfe80b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe80d0e30 0x7fcfe8150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 0x7fcfe80cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80cc260 con 0x7fcfe815c570 2023-10-19T10:52:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80cc6f0 con 0x7fcfe80d0e30 2023-10-19T10:52:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.559+0000 7fcfef707640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8151270 con 0x7fcfe80cd920 2023-10-19T10:52:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 0x7fcfe80cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 0x7fcfe80cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39050/0 (socket says 172.21.15.50:39050) 2023-10-19T10:52:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 -- 172.21.15.50:0/3754128743 learned_addr learned my addr 172.21.15.50:0/3754128743 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfed4a3640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80cd920 0x7fcfe80b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfecca2640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe80d0e30 0x7fcfe8150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 -- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80cd920 msgr2=0x7fcfe80b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80cd920 0x7fcfe80b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 -- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe80d0e30 msgr2=0x7fcfe8150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe80d0e30 0x7fcfe8150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 -- 172.21.15.50:0/3754128743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8009400 con 0x7fcfe815c570 2023-10-19T10:52:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfedca4640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 0x7fcfe80cae00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e7f0 tx=0x7fcfe000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:50.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0009460 con 0x7fcfe815c570 2023-10-19T10:52:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe000c730 con 0x7fcfe815c570 2023-10-19T10:52:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe80cb650 con 0x7fcfe815c570 2023-10-19T10:52:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.560+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000cae0 con 0x7fcfe815c570 2023-10-19T10:52:50.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.561+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe80cbc10 con 0x7fcfe815c570 2023-10-19T10:52:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.561+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe80cc3d0 con 0x7fcfe815c570 2023-10-19T10:52:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.562+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcfe000cc40 con 0x7fcfe815c570 2023-10-19T10:52:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.562+0000 7fcfddffb640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcfcc06a050 0x7fcfcc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.563+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fcfe0088750 con 0x7fcfe815c570 2023-10-19T10:52:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.563+0000 7fcfed4a3640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcfcc06a050 0x7fcfcc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.563+0000 7fcfed4a3640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcfcc06a050 0x7fcfcc06c510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40015b0 tx=0x7fcfe4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.567+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fcfe0053fe0 con 0x7fcfe815c570 2023-10-19T10:52:50.813 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:50.810+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]} v 0) v1 -- 0x7fcfe8073990 con 0x7fcfe815c570 2023-10-19T10:52:51.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.489+0000 7fcfddffb640 1 -- 172.21.15.50:0/3754128743 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]}]=0 v573) v1 ==== 171+0+0 (secure 0 0 0) 0x7fcfe0057600 con 0x7fcfe815c570 2023-10-19T10:52:51.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcfcc06a050 msgr2=0x7fcfcc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:51.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcfcc06a050 0x7fcfcc06c510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40015b0 tx=0x7fcfe4009040 comp rx=0 tx=0).stop 2023-10-19T10:52:51.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 msgr2=0x7fcfe80cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:51.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 0x7fcfe80cae00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e7f0 tx=0x7fcfe000ecb0 comp rx=0 tx=0).stop 2023-10-19T10:52:51.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 shutdown_connections 2023-10-19T10:52:51.496 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fcfcc06a050 0x7fcfcc06c510 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:51.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe815c570 0x7fcfe80cae00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:51.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe80d0e30 0x7fcfe8150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:51.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 --2- 172.21.15.50:0/3754128743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe80cd920 0x7fcfe80b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:51.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 >> 172.21.15.50:0/3754128743 conn(0x7fcfe8089ff0 msgr2=0x7fcfe815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:51.497 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 shutdown_connections 2023-10-19T10:52:51.498 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:51.492+0000 7fcfef707640 1 -- 172.21.15.50:0/3754128743 wait complete. 2023-10-19T10:52:51.509 INFO:tasks.cephfs_test_runner:If any client gets unresponsive/it's session get idle due to lagginess ... ERROR 2023-10-19T10:52:51.509 INFO:tasks.cephfs_test_runner: 2023-10-19T10:52:51.510 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-19T10:52:51.510 INFO:tasks.cephfs_test_runner:ERROR: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-19T10:52:51.510 INFO:tasks.cephfs_test_runner:If any client gets unresponsive/it's session get idle due to lagginess 2023-10-19T10:52:51.511 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-19T10:52:51.511 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-19T10:52:51.511 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa/tasks/cephfs/test_client_recovery.py", line 827, in test_client_eviction_if_config_is_set 2023-10-19T10:52:51.511 INFO:tasks.cephfs_test_runner: while proceed(): 2023-10-19T10:52:51.512 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/contextutil.py", line 134, in __call__ 2023-10-19T10:52:51.512 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2023-10-19T10:52:51.512 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (7) after waiting for 6 seconds 2023-10-19T10:52:51.512 INFO:tasks.cephfs_test_runner: 2023-10-19T10:52:51.513 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-19T10:52:51.513 INFO:tasks.cephfs_test_runner:Ran 2 tests in 904.483s 2023-10-19T10:52:51.513 INFO:tasks.cephfs_test_runner: 2023-10-19T10:52:51.514 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=1) 2023-10-19T10:52:51.514 INFO:tasks.cephfs_test_runner: 2023-10-19T10:52:51.514 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-19T10:52:51.515 INFO:tasks.cephfs_test_runner:ERROR: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-19T10:52:51.515 INFO:tasks.cephfs_test_runner:If any client gets unresponsive/it's session get idle due to lagginess 2023-10-19T10:52:51.515 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-19T10:52:51.515 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-19T10:52:51.516 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa/tasks/cephfs/test_client_recovery.py", line 827, in test_client_eviction_if_config_is_set 2023-10-19T10:52:51.516 INFO:tasks.cephfs_test_runner: while proceed(): 2023-10-19T10:52:51.516 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/contextutil.py", line 134, in __call__ 2023-10-19T10:52:51.516 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2023-10-19T10:52:51.517 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (7) after waiting for 6 seconds 2023-10-19T10:52:51.517 INFO:tasks.cephfs_test_runner: 2023-10-19T10:52:51.517 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-19T10:52:51.585 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7b82acfc2b8145d2a15389135e6c8c02 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_6f3e6b48891b9b50415c0822b6b870797fcda16d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-19T10:52:51.587 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-19T10:52:51.596 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-19T10:52:51.604 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-19T10:52:51.604 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:52:51.605 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:51.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055614,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6496528k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19717 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248260k,nr_inodes=812065,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6496528k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-19T10:52:51.640 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-10-19T10:52:51.640 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:51.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055614,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6496528k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19717 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248260k,nr_inodes=812065,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6496528k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-19T10:52:51.697 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:52:51.698 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:51.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16774 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-19T10:52:51.717 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-19T10:52:51.717 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-19T10:52:51.774 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16774 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-19T10:52:51.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:51.775 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-10-19T10:52:51.832 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-19T10:52:51.833 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-19T10:52:51.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:51.834 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-19T10:52:51.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:51.905 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-19T10:52:51.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:51.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2023-10-19T10:52:52.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:52.087 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2023-10-19T10:52:52.106 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-19T10:52:52.106 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-19T10:52:52.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:52.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-19T10:52:52.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:52.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-19T10:52:52.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-19T10:52:52.258 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2023-10-19T10:52:52.353 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-19T10:52:52.363 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-19T10:52:52.364 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:52:52.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.468+0000 7fd14fedc640 1 Processor -- start 2023-10-19T10:52:52.472 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.469+0000 7fd14fedc640 1 -- start start 2023-10-19T10:52:52.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.469+0000 7fd14fedc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1480d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.469+0000 7fd14fedc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd1480cd920 0x7fd1480d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.469+0000 7fd14fedc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14806aa30 con 0x7fd14815c950 2023-10-19T10:52:52.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.469+0000 7fd14fedc640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14806b2d0 con 0x7fd1480cd920 2023-10-19T10:52:52.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.469+0000 7fd14fedc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480ce090 con 0x7fd14815c570 2023-10-19T10:52:52.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd1480cd920 0x7fd1480d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14dc78640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1480d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd1480cd920 0x7fd1480d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39064/0 (socket says 172.21.15.50:39064) 2023-10-19T10:52:52.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 -- 172.21.15.50:0/470174088 learned_addr learned my addr 172.21.15.50:0/470174088 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:52.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14d477640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd14815c950 0x7fd14806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42644/0 (socket says 172.21.15.50:42644) 2023-10-19T10:52:52.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 -- 172.21.15.50:0/470174088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 msgr2=0x7fd1480d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:52.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 --2- 172.21.15.50:0/470174088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1480d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 -- 172.21.15.50:0/470174088 >> v1:172.21.15.50:6789/0 conn(0x7fd14815c950 legacy=0x7fd14806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.470+0000 7fd14e479640 1 -- 172.21.15.50:0/470174088 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14806ac80 con 0x7fd1480cd920 2023-10-19T10:52:52.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14e479640 1 --2- 172.21.15.50:0/470174088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1480cd920 0x7fd1480d26a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd140009e50 tx=0x7fd14002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2889109a5b1e4764 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:52.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14cc76640 1 -- 172.21.15.50:0/470174088 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14002ca70 con 0x7fd1480cd920 2023-10-19T10:52:52.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14cc76640 1 -- 172.21.15.50:0/470174088 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14002cbd0 con 0x7fd1480cd920 2023-10-19T10:52:52.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14cc76640 1 -- 172.21.15.50:0/470174088 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1400343f0 con 0x7fd1480cd920 2023-10-19T10:52:52.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 -- 172.21.15.50:0/470174088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1480cd920 msgr2=0x7fd1480d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 --2- 172.21.15.50:0/470174088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1480cd920 0x7fd1480d26a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd140009e50 tx=0x7fd14002bed0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 -- 172.21.15.50:0/470174088 shutdown_connections 2023-10-19T10:52:52.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 --2- 172.21.15.50:0/470174088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1480cd920 0x7fd1480d26a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 --2- 172.21.15.50:0/470174088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1480d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 -- 172.21.15.50:0/470174088 >> 172.21.15.50:0/470174088 conn(0x7fd148089ff0 msgr2=0x7fd14809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:52.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 -- 172.21.15.50:0/470174088 shutdown_connections 2023-10-19T10:52:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.471+0000 7fd14fedc640 1 -- 172.21.15.50:0/470174088 wait complete. 2023-10-19T10:52:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 Processor -- start 2023-10-19T10:52:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 -- start start 2023-10-19T10:52:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1480cd920 0x7fd14815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1481599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 0x7fd14815f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480cb3e0 con 0x7fd14815c950 2023-10-19T10:52:52.487 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14815fe70 con 0x7fd1480cd920 2023-10-19T10:52:52.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14fedc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd148159f20 con 0x7fd14815c570 2023-10-19T10:52:52.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14e479640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 0x7fd14815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.473+0000 7fd14e479640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 0x7fd14815f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39066/0 (socket says 172.21.15.50:39066) 2023-10-19T10:52:52.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 -- 172.21.15.50:0/919395125 learned_addr learned my addr 172.21.15.50:0/919395125 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:52.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14d477640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1481599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14dc78640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1480cd920 0x7fd14815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 -- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 msgr2=0x7fd1481599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1481599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 -- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1480cd920 msgr2=0x7fd14815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1480cd920 0x7fd14815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 -- 172.21.15.50:0/919395125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140009b30 con 0x7fd14815c950 2023-10-19T10:52:52.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14d477640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1481599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:52.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14dc78640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1480cd920 0x7fd14815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:52.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14e479640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 0x7fd14815f500 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd140036040 tx=0x7fd140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:52.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140009510 con 0x7fd14815c950 2023-10-19T10:52:52.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14003e450 con 0x7fd14815c950 2023-10-19T10:52:52.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1480caab0 con 0x7fd14815c950 2023-10-19T10:52:52.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14003e800 con 0x7fd14815c950 2023-10-19T10:52:52.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.474+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1480cb070 con 0x7fd14815c950 2023-10-19T10:52:52.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.475+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1480cb590 con 0x7fd14815c950 2023-10-19T10:52:52.493 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.476+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd14003e960 con 0x7fd14815c950 2023-10-19T10:52:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.476+0000 7fd13e7fc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd12c06a050 0x7fd12c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.476+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd1400aafe0 con 0x7fd14815c950 2023-10-19T10:52:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.476+0000 7fd14dc78640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd12c06a050 0x7fd12c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.477+0000 7fd14dc78640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd12c06a050 0x7fd12c06c510 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd144009ab0 tx=0x7fd144008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.480+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fd1480cb590 con 0x7fd14815c950 2023-10-19T10:52:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.653+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1480cddb0 con 0x7fd12c06a050 2023-10-19T10:52:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.656+0000 7fd13e7fc640 1 -- 172.21.15.50:0/919395125 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72891 (secure 0 0 0) 0x7fd144010930 con 0x7fd12c06a050 2023-10-19T10:52:52.660 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:52:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd12c06a050 msgr2=0x7fd12c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd12c06a050 0x7fd12c06c510 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd144009ab0 tx=0x7fd144008040 comp rx=0 tx=0).stop 2023-10-19T10:52:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 msgr2=0x7fd14815f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 0x7fd14815f500 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd140036040 tx=0x7fd140004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 shutdown_connections 2023-10-19T10:52:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd12c06a050 0x7fd12c06c510 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14815c950 0x7fd14815f500 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd14815c570 0x7fd1481599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 --2- 172.21.15.50:0/919395125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1480cd920 0x7fd14815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 >> 172.21.15.50:0/919395125 conn(0x7fd148089ff0 msgr2=0x7fd1480cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 shutdown_connections 2023-10-19T10:52:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.658+0000 7fd14fedc640 1 -- 172.21.15.50:0/919395125 wait complete. 2023-10-19T10:52:52.671 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-10-19T10:52:50.724972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239352,"kb_used_data":6304,"kb_used_omap":17,"kb_used_meta":232942,"kb_avail":754735368,"statfs":{"total":773094113280,"available":772849016832,"internally_reserved":0,"allocated":6455296,"data_stored":3687000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17557,"internal_metadata":238533483},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12366,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-58,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001443"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-19T10:52:46.438215+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:52:46.438215+0000","last_peered":"2023-10-19T10:52:46.438215+0000","last_clean":"2023-10-19T10:52:46.438215+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:52:46.438215+0000","last_undegraded":"2023-10-19T10:52:46.438215+0000","last_fullsized":"2023-10-19T10:52:46.438215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:28:49.007377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-19T10:49:51.562730+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:49:51.562730+0000","last_peered":"2023-10-19T10:49:51.562730+0000","last_clean":"2023-10-19T10:49:51.562730+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:49:51.562730+0000","last_undegraded":"2023-10-19T10:49:51.562730+0000","last_fullsized":"2023-10-19T10:49:51.562730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:41:16.213420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.484056+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:52:48.484056+0000","last_peered":"2023-10-19T10:52:48.484056+0000","last_clean":"2023-10-19T10:52:48.484056+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:52:48.484056+0000","last_undegraded":"2023-10-19T10:52:48.484056+0000","last_fullsized":"2023-10-19T10:52:48.484056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:10:34.155282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-19T10:49:21.559265+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:49:21.559265+0000","last_peered":"2023-10-19T10:49:21.559265+0000","last_clean":"2023-10-19T10:49:21.559265+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:49:21.559265+0000","last_undegraded":"2023-10-19T10:49:21.559265+0000","last_fullsized":"2023-10-19T10:49:21.559265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:39:51.092152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-19T10:49:21.560302+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:49:21.560302+0000","last_peered":"2023-10-19T10:49:21.560302+0000","last_clean":"2023-10-19T10:49:21.560302+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:49:21.560302+0000","last_undegraded":"2023-10-19T10:49:21.560302+0000","last_fullsized":"2023-10-19T10:49:21.560302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:48:31.831362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.938373+0000","last_change":"2023-10-19T10:46:26.328460+0000","last_active":"2023-10-19T10:52:48.938373+0000","last_peered":"2023-10-19T10:52:48.938373+0000","last_clean":"2023-10-19T10:52:48.938373+0000","last_became_active":"2023-10-19T10:46:26.328144+0000","last_became_peered":"2023-10-19T10:46:26.328144+0000","last_unstale":"2023-10-19T10:52:48.938373+0000","last_undegraded":"2023-10-19T10:52:48.938373+0000","last_fullsized":"2023-10-19T10:52:48.938373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:22:19.029544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.482796+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:52:48.482796+0000","last_peered":"2023-10-19T10:52:48.482796+0000","last_clean":"2023-10-19T10:52:48.482796+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:52:48.482796+0000","last_undegraded":"2023-10-19T10:52:48.482796+0000","last_fullsized":"2023-10-19T10:52:48.482796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:02:39.951336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-19T10:49:21.560320+0000","last_change":"2023-10-19T10:46:26.329138+0000","last_active":"2023-10-19T10:49:21.560320+0000","last_peered":"2023-10-19T10:49:21.560320+0000","last_clean":"2023-10-19T10:49:21.560320+0000","last_became_active":"2023-10-19T10:46:26.328870+0000","last_became_peered":"2023-10-19T10:46:26.328870+0000","last_unstale":"2023-10-19T10:49:21.560320+0000","last_undegraded":"2023-10-19T10:49:21.560320+0000","last_fullsized":"2023-10-19T10:49:21.560320+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:29:41.970449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":249,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.483400+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:52:48.483400+0000","last_peered":"2023-10-19T10:52:48.483400+0000","last_clean":"2023-10-19T10:52:48.483400+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:52:48.483400+0000","last_undegraded":"2023-10-19T10:52:48.483400+0000","last_fullsized":"2023-10-19T10:52:48.483400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:18:37.139568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542341,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1253376,"data_stored":905845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.88400000000000001,"15min":0.91300000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61399999999999999,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":0.80000000000000004,"15min":0.84599999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.53500000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.754,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Oct 19 10:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.54100000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73599999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.88,"15min":0.88},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.94499999999999995},"last":0.502}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.28699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.313,"15min":0.28999999999999998},"max":{"1min":0.76600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.28000000000000003},"max":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.84199999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.872,"15min":0.872},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.78100000000000003,"5min":0.78300000000000003,"15min":0.83599999999999997},"last":0.40100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":655360,"data_stored":309247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.89900000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":0.84599999999999997,"15min":0.90700000000000003},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.313},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.83499999999999996},"last":0.628},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.34499999999999997},"max":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.88},"last":0.502}]},{"osd":2,"last update":"Thu Oct 19 10:52:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":1.042,"15min":1.042},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.34499999999999997},"max":{"1min":0.77200000000000002,"5min":1.075,"15min":1.075},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.317,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":1.111,"15min":1.111},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48999999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68200000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48699999999999999,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.499,"15min":0.50800000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":0.92600000000000005,"15min":1.02},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":647168,"data_stored":308544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.312},"max":{"1min":0.747,"5min":0.81599999999999995,"15min":0.997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.32000000000000001},"max":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.93600000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.33400000000000002},"max":{"1min":0.76100000000000001,"5min":0.83499999999999996,"15min":0.87},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:52:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.752,"5min":0.752,"15min":0.79500000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.85599999999999998,"15min":0.91300000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.33800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.27900000000000003},"max":{"1min":0.66800000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.45900000000000002,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.26600000000000001},"max":{"1min":0.65200000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34300000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.84199999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80000000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.89000000000000001,"15min":0.89600000000000002},"last":0.52100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":667648,"data_stored":321083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78400000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.33800000000000002},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.83299999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71899999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"last":0.437}]},{"osd":3,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40100000000000002},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.27800000000000002},"max":{"1min":0.753,"5min":0.75600000000000001,"15min":0.879},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.33800000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.94799999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.91100000000000003,"15min":0.97099999999999997},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.51000000000000001},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.86199999999999999,"15min":0.96499999999999997},"last":0.54400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":655360,"data_stored":309112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.182},"max":{"1min":0.54800000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.22},"max":{"1min":0.60399999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.317}]},{"osd":1,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42099999999999999,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.247,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.69199999999999995,"15min":0.72199999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.222},"max":{"1min":0.56999999999999995,"5min":0.69399999999999995,"15min":0.74099999999999999},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46000000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.69999999999999996},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70099999999999996,"15min":0.68600000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.36099999999999999},"max":{"1min":0.90300000000000002,"5min":0.92700000000000005,"15min":1.0149999999999999},"last":0.504},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.66800000000000004},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.376},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.35699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.39000000000000001},"max":{"1min":0.90300000000000002,"5min":0.93700000000000006,"15min":0.997},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.36699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.96099999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.40000000000000002},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":1.0349999999999999},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:52:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69399999999999995,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.98699999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53200000000000003}]}]},{"osd":2,"up_from":54,"seq":231928234062,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":660,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":675840,"data_stored":323364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5090,"internal_metadata":32369694},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.376,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.223},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.432,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":0.70499999999999996},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.309},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46700000000000003,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":1.006,"15min":1.006},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.67900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79700000000000004,"5min":1.05,"15min":1.05},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":0.93999999999999995},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.96099999999999997,"15min":0.97099999999999997},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":1.006},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95199999999999996,"5min":1.028,"15min":1.028},"last":0.46200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":655360,"data_stored":309787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.219},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":1.018},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.88500000000000001},"last":0.505}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.437},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.435,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.90800000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.76800000000000002,"5min":0.82299999999999995,"15min":0.83599999999999997},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.41899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":1.0740000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.503,"5min":0.42999999999999999,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.96799999999999997},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.1839999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.66000000000000003},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.433},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70499999999999996,"15min":0.66800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48299999999999998,"15min":0.41099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41099999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.45000000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.224},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1245184,"data_stored":900018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.83599999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.20300000000000001},"max":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":1.077},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58799999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.314},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59299999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.28699999999999998,"15min":0.221},"max":{"1min":0.64500000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.247},"max":{"1min":0.625,"5min":0.74099999999999999,"15min":0.745},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.47299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.02,"15min":1.1499999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93600000000000005},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.99099999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.40200000000000002},"max":{"1min":0.90100000000000002,"5min":0.96299999999999997,"15min":1.1919999999999999},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":1.0029999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.1910000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.57999999999999996,"5min":0.435,"15min":0.39800000000000002},"max":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":1.0289999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0960000000000001},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:52:52.674 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:52:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.777+0000 7f352a0ab640 1 Processor -- start 2023-10-19T10:52:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.777+0000 7f352a0ab640 1 -- start start 2023-10-19T10:52:52.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f352a0ab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3524160340 0x7f35241607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f352a0ab640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f352415ddf0 0x7f352415e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f352a0ab640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352415d6e0 con 0x7f35240b1f30 2023-10-19T10:52:52.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f352a0ab640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352415f030 con 0x7f352415ddf0 2023-10-19T10:52:52.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f352a0ab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352415cdb0 con 0x7f3524160340 2023-10-19T10:52:52.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f3523fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f352415ddf0 0x7f352415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f35237fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f35240b1f30 0x7f35240b2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42652/0 (socket says 172.21.15.50:42652) 2023-10-19T10:52:52.785 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f35237fe640 1 -- 172.21.15.50:0/3506308080 learned_addr learned my addr 172.21.15.50:0/3506308080 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.778+0000 7f3523fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f352415ddf0 0x7f352415e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39068/0 (socket says 172.21.15.50:39068) 2023-10-19T10:52:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f351bfff640 1 --2- 172.21.15.50:0/3506308080 >> v2:172.21.15.183:3300/0 conn(0x7f3524160340 0x7f35241607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f3523fff640 1 -- 172.21.15.50:0/3506308080 >> v2:172.21.15.183:3300/0 conn(0x7f3524160340 msgr2=0x7f35241607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:52.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f3523fff640 1 --2- 172.21.15.50:0/3506308080 >> v2:172.21.15.183:3300/0 conn(0x7f3524160340 0x7f35241607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f3523fff640 1 -- 172.21.15.50:0/3506308080 >> v1:172.21.15.50:6789/0 conn(0x7f35240b1f30 legacy=0x7f35240b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f3523fff640 1 -- 172.21.15.50:0/3506308080 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352415d090 con 0x7f352415ddf0 2023-10-19T10:52:52.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f3522ffd640 1 -- 172.21.15.50:0/3506308080 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474354237 0 0) 0x7f352415d6e0 con 0x7f35240b1f30 2023-10-19T10:52:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f351bfff640 1 --2- 172.21.15.50:0/3506308080 >> v2:172.21.15.183:3300/0 conn(0x7f3524160340 0x7f35241607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.779+0000 7f3523fff640 1 --2- 172.21.15.50:0/3506308080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352415ddf0 0x7f352415e280 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f35240cfbf0 tx=0x7f351c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1497589168c7cf server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f3522ffd640 1 -- 172.21.15.50:0/3506308080 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c02cbf0 con 0x7f352415ddf0 2023-10-19T10:52:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f3522ffd640 1 -- 172.21.15.50:0/3506308080 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c02cd50 con 0x7f352415ddf0 2023-10-19T10:52:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f3522ffd640 1 -- 172.21.15.50:0/3506308080 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c034660 con 0x7f352415ddf0 2023-10-19T10:52:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 -- 172.21.15.50:0/3506308080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352415ddf0 msgr2=0x7f352415e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 --2- 172.21.15.50:0/3506308080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352415ddf0 0x7f352415e280 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f35240cfbf0 tx=0x7f351c02be40 comp rx=0 tx=0).stop 2023-10-19T10:52:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 -- 172.21.15.50:0/3506308080 shutdown_connections 2023-10-19T10:52:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 --2- 172.21.15.50:0/3506308080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352415ddf0 0x7f352415e280 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 --2- 172.21.15.50:0/3506308080 >> v2:172.21.15.183:3300/0 conn(0x7f3524160340 0x7f35241607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 -- 172.21.15.50:0/3506308080 >> 172.21.15.50:0/3506308080 conn(0x7f35240ccf30 msgr2=0x7f35240cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 -- 172.21.15.50:0/3506308080 shutdown_connections 2023-10-19T10:52:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.780+0000 7f352a0ab640 1 -- 172.21.15.50:0/3506308080 wait complete. 2023-10-19T10:52:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.781+0000 7f352a0ab640 1 Processor -- start 2023-10-19T10:52:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.781+0000 7f352a0ab640 1 -- start start 2023-10-19T10:52:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f352a0ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35240b1f30 0x7f35241f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f352a0ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352415ddf0 0x7f35240c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f352a0ab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 0x7f35240b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f352a0ab640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35240b4a90 con 0x7f3524160340 2023-10-19T10:52:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f352a0ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35240c8a50 con 0x7f352415ddf0 2023-10-19T10:52:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f352a0ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35240b3f10 con 0x7f35240b1f30 2023-10-19T10:52:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 0x7f35240b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 0x7f35240b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39084/0 (socket says 172.21.15.50:39084) 2023-10-19T10:52:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 -- 172.21.15.50:0/936699035 learned_addr learned my addr 172.21.15.50:0/936699035 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f35237fe640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35240b1f30 0x7f35241f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f351bfff640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352415ddf0 0x7f35240c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 -- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35240b1f30 msgr2=0x7f35241f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35240b1f30 0x7f35241f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 -- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352415ddf0 msgr2=0x7f35240c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352415ddf0 0x7f35240c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f3523fff640 1 -- 172.21.15.50:0/936699035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c009a70 con 0x7f3524160340 2023-10-19T10:52:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.782+0000 7f35237fe640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35240b1f30 0x7f35241f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.783+0000 7f3523fff640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 0x7f35240b51a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f351c02c450 tx=0x7f351c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.783+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c03e610 con 0x7f3524160340 2023-10-19T10:52:52.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.783+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c03e770 con 0x7f3524160340 2023-10-19T10:52:52.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.783+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35240b41f0 con 0x7f3524160340 2023-10-19T10:52:52.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.783+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c03eb20 con 0x7f3524160340 2023-10-19T10:52:52.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.783+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35240b47b0 con 0x7f3524160340 2023-10-19T10:52:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.784+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35240c8c50 con 0x7f3524160340 2023-10-19T10:52:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.785+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f351c03b050 con 0x7f3524160340 2023-10-19T10:52:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.785+0000 7f3520ff9640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f350406a050 0x7f350406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.785+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f351c0aa870 con 0x7f3524160340 2023-10-19T10:52:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.785+0000 7f35237fe640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f350406a050 0x7f350406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.786+0000 7f35237fe640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f350406a050 0x7f350406c510 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3514000f60 tx=0x7f3514007cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.789+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f35240c8c50 con 0x7f3524160340 2023-10-19T10:52:52.963 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.960+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f352415fd20 con 0x7f350406a050 2023-10-19T10:52:52.966 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.964+0000 7f3520ff9640 1 -- 172.21.15.50:0/936699035 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72961 (secure 0 0 0) 0x7f3514005d40 con 0x7f350406a050 2023-10-19T10:52:52.967 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:52.968 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:52:52.969 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f350406a050 msgr2=0x7f350406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f350406a050 0x7f350406c510 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3514000f60 tx=0x7f3514007cc0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 msgr2=0x7f35240b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:52.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 0x7f35240b51a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f351c02c450 tx=0x7f351c004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:52.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 shutdown_connections 2023-10-19T10:52:52.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f350406a050 0x7f350406c510 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3524160340 0x7f35240b51a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352415ddf0 0x7f35240c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.965+0000 7f352a0ab640 1 --2- 172.21.15.50:0/936699035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35240b1f30 0x7f35241f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:52.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.966+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 >> 172.21.15.50:0/936699035 conn(0x7f35240ccf30 msgr2=0x7f35240cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:52.972 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.966+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 shutdown_connections 2023-10-19T10:52:52.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:52.966+0000 7f352a0ab640 1 -- 172.21.15.50:0/936699035 wait complete. 2023-10-19T10:52:52.979 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2023-10-19T10:52:52.725328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239580,"kb_used_data":6508,"kb_used_omap":17,"kb_used_meta":232942,"kb_avail":754735140,"statfs":{"total":773094113280,"available":772848783360,"internally_reserved":0,"allocated":6664192,"data_stored":3794225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17557,"internal_metadata":238533483},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12366,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-58,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.288616"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.481478+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:52:49.481478+0000","last_peered":"2023-10-19T10:52:49.481478+0000","last_clean":"2023-10-19T10:52:49.481478+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:52:49.481478+0000","last_undegraded":"2023-10-19T10:52:49.481478+0000","last_fullsized":"2023-10-19T10:52:49.481478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:50.979513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.077435+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:52:50.077435+0000","last_peered":"2023-10-19T10:52:50.077435+0000","last_clean":"2023-10-19T10:52:50.077435+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:52:50.077435+0000","last_undegraded":"2023-10-19T10:52:50.077435+0000","last_fullsized":"2023-10-19T10:52:50.077435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:17:53.220183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.484056+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:52:48.484056+0000","last_peered":"2023-10-19T10:52:48.484056+0000","last_clean":"2023-10-19T10:52:48.484056+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:52:48.484056+0000","last_undegraded":"2023-10-19T10:52:48.484056+0000","last_fullsized":"2023-10-19T10:52:48.484056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:10:34.155282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.075149+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:52:50.075149+0000","last_peered":"2023-10-19T10:52:50.075149+0000","last_clean":"2023-10-19T10:52:50.075149+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:52:50.075149+0000","last_undegraded":"2023-10-19T10:52:50.075149+0000","last_fullsized":"2023-10-19T10:52:50.075149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:31:36.289755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.075926+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:52:50.075926+0000","last_peered":"2023-10-19T10:52:50.075926+0000","last_clean":"2023-10-19T10:52:50.075926+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:52:50.075926+0000","last_undegraded":"2023-10-19T10:52:50.075926+0000","last_fullsized":"2023-10-19T10:52:50.075926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:00.529540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.938373+0000","last_change":"2023-10-19T10:46:26.328460+0000","last_active":"2023-10-19T10:52:48.938373+0000","last_peered":"2023-10-19T10:52:48.938373+0000","last_clean":"2023-10-19T10:52:48.938373+0000","last_became_active":"2023-10-19T10:46:26.328144+0000","last_became_peered":"2023-10-19T10:46:26.328144+0000","last_unstale":"2023-10-19T10:52:48.938373+0000","last_undegraded":"2023-10-19T10:52:48.938373+0000","last_fullsized":"2023-10-19T10:52:48.938373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:22:19.029544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.482796+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:52:48.482796+0000","last_peered":"2023-10-19T10:52:48.482796+0000","last_clean":"2023-10-19T10:52:48.482796+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:52:48.482796+0000","last_undegraded":"2023-10-19T10:52:48.482796+0000","last_fullsized":"2023-10-19T10:52:48.482796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:02:39.951336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.076673+0000","last_change":"2023-10-19T10:46:26.329138+0000","last_active":"2023-10-19T10:52:50.076673+0000","last_peered":"2023-10-19T10:52:50.076673+0000","last_clean":"2023-10-19T10:52:50.076673+0000","last_became_active":"2023-10-19T10:46:26.328870+0000","last_became_peered":"2023-10-19T10:46:26.328870+0000","last_unstale":"2023-10-19T10:52:50.076673+0000","last_undegraded":"2023-10-19T10:52:50.076673+0000","last_fullsized":"2023-10-19T10:52:50.076673+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:01.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":249,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.483400+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:52:48.483400+0000","last_peered":"2023-10-19T10:52:48.483400+0000","last_clean":"2023-10-19T10:52:48.483400+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:52:48.483400+0000","last_undegraded":"2023-10-19T10:52:48.483400+0000","last_fullsized":"2023-10-19T10:52:48.483400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:18:37.139568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542341,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1253376,"data_stored":905845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.88400000000000001,"15min":0.91300000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61399999999999999,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":0.80000000000000004,"15min":0.84599999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.53500000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.754,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Oct 19 10:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.54100000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73599999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.88,"15min":0.88},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.94499999999999995},"last":0.502}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.28699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.313,"15min":0.28999999999999998},"max":{"1min":0.76600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.28000000000000003},"max":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.84199999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.872,"15min":0.872},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.78100000000000003,"5min":0.78300000000000003,"15min":0.83599999999999997},"last":0.40100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.89900000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":0.84599999999999997,"15min":0.90700000000000003},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.313},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.83499999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.34499999999999997},"max":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.88},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:52:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":1.042,"15min":1.042},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.34499999999999997},"max":{"1min":0.77200000000000002,"5min":1.075,"15min":1.075},"last":0.44},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.317,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":1.111,"15min":1.111},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48999999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68200000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48699999999999999,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.499,"15min":0.50800000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":0.92600000000000005,"15min":1.02},"last":0.70099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.312},"max":{"1min":0.747,"5min":0.81599999999999995,"15min":0.997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.32000000000000001},"max":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.93600000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.33400000000000002},"max":{"1min":0.76100000000000001,"5min":0.83499999999999996,"15min":0.87},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:52:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.752,"5min":0.752,"15min":0.79500000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.85599999999999998,"15min":0.91300000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.33800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.27900000000000003},"max":{"1min":0.66800000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.45900000000000002,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.26600000000000001},"max":{"1min":0.65200000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.84199999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80000000000000004},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.89000000000000001,"15min":0.89600000000000002},"last":0.432}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78400000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.33800000000000002},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.89700000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.83299999999999996},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71899999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"last":0.621}]},{"osd":3,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40100000000000002},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.746}]},{"osd":5,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.27800000000000002},"max":{"1min":0.753,"5min":0.75600000000000001,"15min":0.879},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.94799999999999995},"last":0.496},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.91100000000000003,"15min":0.97099999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.51000000000000001},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.86199999999999999,"15min":0.96499999999999997},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.182},"max":{"1min":0.54800000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.22},"max":{"1min":0.60399999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.247}]},{"osd":1,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42099999999999999,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.247,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.69199999999999995,"15min":0.72199999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.222},"max":{"1min":0.56999999999999995,"5min":0.69399999999999995,"15min":0.74099999999999999},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46000000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.69999999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70099999999999996,"15min":0.68600000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.36099999999999999},"max":{"1min":0.90300000000000002,"5min":0.92700000000000005,"15min":1.0149999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.66800000000000004},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.376},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.35699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.39000000000000001},"max":{"1min":0.90300000000000002,"5min":0.93700000000000006,"15min":0.997},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.36699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.96099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.40000000000000002},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":1.0349999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:52:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69399999999999995,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.98699999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65500000000000003}]}]},{"osd":2,"up_from":54,"seq":231928234062,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":660,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":675840,"data_stored":323364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5090,"internal_metadata":32369694},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.376,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.223},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.432,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":0.70499999999999996},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.309},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46700000000000003,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":1.006,"15min":1.006},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.67900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79700000000000004,"5min":1.05,"15min":1.05},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":0.93999999999999995},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.96099999999999997,"15min":0.97099999999999997},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":1.006},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95199999999999996,"5min":1.028,"15min":1.028},"last":0.46200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.219},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":1.018},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.88500000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.505},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.373}]},{"osd":3,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.435,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.90800000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.76800000000000002,"5min":0.82299999999999995,"15min":0.83599999999999997},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.41899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":1.0740000000000001},"last":0.622},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.503,"5min":0.42999999999999999,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.96799999999999997},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.1839999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.66000000000000003},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.433},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.70499999999999996,"15min":0.66800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48299999999999998,"15min":0.41099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41099999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.45000000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.224},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575046,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1286144,"data_stored":920047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.83599999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.20300000000000001},"max":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":1.077},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Oct 19 10:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58799999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59299999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.28699999999999998,"15min":0.221},"max":{"1min":0.64500000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.247},"max":{"1min":0.625,"5min":0.74099999999999999,"15min":0.745},"last":0.373}]},{"osd":4,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.47299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.02,"15min":1.1499999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93600000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.99099999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.40200000000000002},"max":{"1min":0.90100000000000002,"5min":0.96299999999999997,"15min":1.1919999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":1.0029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.1910000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.57999999999999996,"5min":0.435,"15min":0.39800000000000002},"max":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":1.0289999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0960000000000001},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:52:52.981 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-19T10:52:52.982 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:52:53.090 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.087+0000 7f2f878c4640 1 Processor -- start 2023-10-19T10:52:53.091 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f878c4640 1 -- start start 2023-10-19T10:52:53.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f878c4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2f800b23c0 0x7f2f800ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f878c4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800caf80 con 0x7f2f8015a370 2023-10-19T10:52:53.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f878c4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800d02e0 con 0x7f2f800b1f30 2023-10-19T10:52:53.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f878c4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f800b2850 con 0x7f2f800b23c0 2023-10-19T10:52:53.095 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f85660640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2f8015a370 0x7f2f8006fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42668/0 (socket says 172.21.15.50:42668) 2023-10-19T10:52:53.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.088+0000 7f2f85660640 1 -- 172.21.15.50:0/4228031183 learned_addr learned my addr 172.21.15.50:0/4228031183 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:53.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f85e61640 1 --2- 172.21.15.50:0/4228031183 >> v2:172.21.15.183:3300/0 conn(0x7f2f800b23c0 0x7f2f800ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310468004 0 0) 0x7f2f800caf80 con 0x7f2f8015a370 2023-10-19T10:52:53.096 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f6c003520 con 0x7f2f8015a370 2023-10-19T10:52:53.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 399401765 0 0) 0x7f2f6c003520 con 0x7f2f8015a370 2023-10-19T10:52:53.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 >> v2:172.21.15.183:3300/0 conn(0x7f2f800b23c0 msgr2=0x7f2f800ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:53.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 --2- 172.21.15.50:0/4228031183 >> v2:172.21.15.183:3300/0 conn(0x7f2f800b23c0 0x7f2f800ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 >> v1:172.21.15.183:6789/0 conn(0x7f2f800b1f30 legacy=0x7f2f800ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8015a750 con 0x7f2f8015a370 2023-10-19T10:52:53.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f2f7c001670 con 0x7f2f8015a370 2023-10-19T10:52:53.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 >> v1:172.21.15.50:6789/0 conn(0x7f2f8015a370 legacy=0x7f2f8006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f6c003970 0x7f2f6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f6c005180 0x7f2f6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800b2850 con 0x7f2f6c0047c0 2023-10-19T10:52:53.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c00c020 con 0x7f2f6c003970 2023-10-19T10:52:53.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c00c1e0 con 0x7f2f6c005180 2023-10-19T10:52:53.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f85660640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f6c003970 0x7f2f6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f85e61640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f6c005180 0x7f2f6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f84e5f640 1 -- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f6c005180 msgr2=0x7f2f6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:53.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f6c005180 0x7f2f6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f84e5f640 1 -- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f6c003970 msgr2=0x7f2f6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:53.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f6c003970 0x7f2f6c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f84e5f640 1 -- 172.21.15.50:0/4228031183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c003520 con 0x7f2f6c0047c0 2023-10-19T10:52:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.089+0000 7f2f85e61640 1 -- 172.21.15.50:0/4228031183 reap_dead start 2023-10-19T10:52:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.090+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f6c004c40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2f70001750 tx=0x7f2f6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ea0c955019816a server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.090+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700052d0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.090+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7002e050 con 0x7f2f6c0047c0 2023-10-19T10:52:53.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.090+0000 7f2f77fff640 1 -- 172.21.15.50:0/4228031183 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70003830 con 0x7f2f6c0047c0 2023-10-19T10:52:53.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 -- 172.21.15.50:0/4228031183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 msgr2=0x7f2f6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f6c004c40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2f70001750 tx=0x7f2f6c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 -- 172.21.15.50:0/4228031183 shutdown_connections 2023-10-19T10:52:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4228031183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f6c004c40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 -- 172.21.15.50:0/4228031183 >> 172.21.15.50:0/4228031183 conn(0x7f2f800ccf30 msgr2=0x7f2f800cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:53.105 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 -- 172.21.15.50:0/4228031183 shutdown_connections 2023-10-19T10:52:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.091+0000 7f2f878c4640 1 -- 172.21.15.50:0/4228031183 wait complete. 2023-10-19T10:52:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.092+0000 7f2f878c4640 1 Processor -- start 2023-10-19T10:52:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.092+0000 7f2f878c4640 1 -- start start 2023-10-19T10:52:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f878c4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f8015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f878c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f800c7c70 0x7f2f80150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f878c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f800b4260 0x7f2f800b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f878c4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800b4c20 con 0x7f2f6c0047c0 2023-10-19T10:52:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f878c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8015c6e0 con 0x7f2f800b4260 2023-10-19T10:52:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f878c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8015c9c0 con 0x7f2f800c7c70 2023-10-19T10:52:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f8015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f8015beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39114/0 (socket says 172.21.15.50:39114) 2023-10-19T10:52:53.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 -- 172.21.15.50:0/4181757679 learned_addr learned my addr 172.21.15.50:0/4181757679 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f800c7c70 0x7f2f80150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85e61640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f800b4260 0x7f2f800b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 -- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f800c7c70 msgr2=0x7f2f80150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f800c7c70 0x7f2f80150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 -- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f800b4260 msgr2=0x7f2f800b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f800b4260 0x7f2f800b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 -- 172.21.15.50:0/4181757679 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8015cca0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f800c7c70 0x7f2f80150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:53.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85e61640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f800b4260 0x7f2f800b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:53.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f85660640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f8015beb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00e780 tx=0x7f2f7c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:53.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.093+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00c6c0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.094+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c00c860 con 0x7f2f6c0047c0 2023-10-19T10:52:53.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.094+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8015cf80 con 0x7f2f6c0047c0 2023-10-19T10:52:53.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.094+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00cc10 con 0x7f2f6c0047c0 2023-10-19T10:52:53.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.094+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8015c3f0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.095+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8015d4a0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.095+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f7c01d460 con 0x7f2f6c0047c0 2023-10-19T10:52:53.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.095+0000 7f2f75ffb640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f6406a080 0x7f2f6406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.095+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2f7c0883b0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.095+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f6406a080 0x7f2f6406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.096+0000 7f2f84e5f640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f6406a080 0x7f2f6406c540 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f70033000 tx=0x7f2f70009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:53.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.100+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f2f8015d4a0 con 0x7f2f6c0047c0 2023-10-19T10:52:53.277 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.274+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f800d0000 con 0x7f2f6406a080 2023-10-19T10:52:53.280 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.277+0000 7f2f75ffb640 1 -- 172.21.15.50:0/4181757679 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72961 (secure 0 0 0) 0x7f2f70007040 con 0x7f2f6406a080 2023-10-19T10:52:53.281 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:53.281 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:52:53.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f6406a080 msgr2=0x7f2f6406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.283 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f6406a080 0x7f2f6406c540 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f70033000 tx=0x7f2f70009030 comp rx=0 tx=0).stop 2023-10-19T10:52:53.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 msgr2=0x7f2f8015beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f8015beb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00e780 tx=0x7f2f7c00ec40 comp rx=0 tx=0).stop 2023-10-19T10:52:53.284 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 shutdown_connections 2023-10-19T10:52:53.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f2f6406a080 0x7f2f6406c540 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f800b4260 0x7f2f800b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f800c7c70 0x7f2f80150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.285 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 --2- 172.21.15.50:0/4181757679 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f6c0047c0 0x7f2f8015beb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 >> 172.21.15.50:0/4181757679 conn(0x7f2f800ccf30 msgr2=0x7f2f800af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:53.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 shutdown_connections 2023-10-19T10:52:53.286 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.279+0000 7f2f878c4640 1 -- 172.21.15.50:0/4181757679 wait complete. 2023-10-19T10:52:53.293 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2023-10-19T10:52:52.725328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239580,"kb_used_data":6508,"kb_used_omap":17,"kb_used_meta":232942,"kb_avail":754735140,"statfs":{"total":773094113280,"available":772848783360,"internally_reserved":0,"allocated":6664192,"data_stored":3794225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17557,"internal_metadata":238533483},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12366,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-58,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.288616"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.481478+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:52:49.481478+0000","last_peered":"2023-10-19T10:52:49.481478+0000","last_clean":"2023-10-19T10:52:49.481478+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:52:49.481478+0000","last_undegraded":"2023-10-19T10:52:49.481478+0000","last_fullsized":"2023-10-19T10:52:49.481478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:50.979513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.077435+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:52:50.077435+0000","last_peered":"2023-10-19T10:52:50.077435+0000","last_clean":"2023-10-19T10:52:50.077435+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:52:50.077435+0000","last_undegraded":"2023-10-19T10:52:50.077435+0000","last_fullsized":"2023-10-19T10:52:50.077435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:17:53.220183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.484056+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:52:48.484056+0000","last_peered":"2023-10-19T10:52:48.484056+0000","last_clean":"2023-10-19T10:52:48.484056+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:52:48.484056+0000","last_undegraded":"2023-10-19T10:52:48.484056+0000","last_fullsized":"2023-10-19T10:52:48.484056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:10:34.155282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.075149+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:52:50.075149+0000","last_peered":"2023-10-19T10:52:50.075149+0000","last_clean":"2023-10-19T10:52:50.075149+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:52:50.075149+0000","last_undegraded":"2023-10-19T10:52:50.075149+0000","last_fullsized":"2023-10-19T10:52:50.075149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:31:36.289755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.075926+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:52:50.075926+0000","last_peered":"2023-10-19T10:52:50.075926+0000","last_clean":"2023-10-19T10:52:50.075926+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:52:50.075926+0000","last_undegraded":"2023-10-19T10:52:50.075926+0000","last_fullsized":"2023-10-19T10:52:50.075926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:00.529540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.938373+0000","last_change":"2023-10-19T10:46:26.328460+0000","last_active":"2023-10-19T10:52:48.938373+0000","last_peered":"2023-10-19T10:52:48.938373+0000","last_clean":"2023-10-19T10:52:48.938373+0000","last_became_active":"2023-10-19T10:46:26.328144+0000","last_became_peered":"2023-10-19T10:46:26.328144+0000","last_unstale":"2023-10-19T10:52:48.938373+0000","last_undegraded":"2023-10-19T10:52:48.938373+0000","last_fullsized":"2023-10-19T10:52:48.938373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:22:19.029544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.482796+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:52:48.482796+0000","last_peered":"2023-10-19T10:52:48.482796+0000","last_clean":"2023-10-19T10:52:48.482796+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:52:48.482796+0000","last_undegraded":"2023-10-19T10:52:48.482796+0000","last_fullsized":"2023-10-19T10:52:48.482796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:02:39.951336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.076673+0000","last_change":"2023-10-19T10:46:26.329138+0000","last_active":"2023-10-19T10:52:50.076673+0000","last_peered":"2023-10-19T10:52:50.076673+0000","last_clean":"2023-10-19T10:52:50.076673+0000","last_became_active":"2023-10-19T10:46:26.328870+0000","last_became_peered":"2023-10-19T10:46:26.328870+0000","last_unstale":"2023-10-19T10:52:50.076673+0000","last_undegraded":"2023-10-19T10:52:50.076673+0000","last_fullsized":"2023-10-19T10:52:50.076673+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:01.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":249,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-19T10:52:48.483400+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:52:48.483400+0000","last_peered":"2023-10-19T10:52:48.483400+0000","last_clean":"2023-10-19T10:52:48.483400+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:52:48.483400+0000","last_undegraded":"2023-10-19T10:52:48.483400+0000","last_fullsized":"2023-10-19T10:52:48.483400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:18:37.139568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542341,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1253376,"data_stored":905845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.88400000000000001,"15min":0.91300000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61399999999999999,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":0.80000000000000004,"15min":0.84599999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.53500000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.754,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Oct 19 10:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.54100000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73599999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.88,"15min":0.88},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.94499999999999995},"last":0.502}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.28699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.313,"15min":0.28999999999999998},"max":{"1min":0.76600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.28000000000000003},"max":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.84199999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.872,"15min":0.872},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.78100000000000003,"5min":0.78300000000000003,"15min":0.83599999999999997},"last":0.40100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.89900000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":0.84599999999999997,"15min":0.90700000000000003},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.313},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.83499999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.34499999999999997},"max":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.88},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:52:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":1.042,"15min":1.042},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.34499999999999997},"max":{"1min":0.77200000000000002,"5min":1.075,"15min":1.075},"last":0.44},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.317,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":1.111,"15min":1.111},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48999999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68200000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48699999999999999,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.499,"15min":0.50800000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":0.92600000000000005,"15min":1.02},"last":0.70099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.312},"max":{"1min":0.747,"5min":0.81599999999999995,"15min":0.997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.32000000000000001},"max":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.93600000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.33400000000000002},"max":{"1min":0.76100000000000001,"5min":0.83499999999999996,"15min":0.87},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Oct 19 10:52:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.752,"5min":0.752,"15min":0.79500000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.85599999999999998,"15min":0.91300000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.33800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.27900000000000003},"max":{"1min":0.66800000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.45900000000000002,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.26600000000000001},"max":{"1min":0.65200000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.84199999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80000000000000004},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.89000000000000001,"15min":0.89600000000000002},"last":0.432}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78400000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.33800000000000002},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.89700000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.83299999999999996},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71899999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"last":0.621}]},{"osd":3,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40100000000000002},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.746}]},{"osd":5,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.27800000000000002},"max":{"1min":0.753,"5min":0.75600000000000001,"15min":0.879},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.94799999999999995},"last":0.496},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.91100000000000003,"15min":0.97099999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.51000000000000001},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.86199999999999999,"15min":0.96499999999999997},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.182},"max":{"1min":0.54800000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.22},"max":{"1min":0.60399999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.247}]},{"osd":1,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42099999999999999,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.247,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.69199999999999995,"15min":0.72199999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.222},"max":{"1min":0.56999999999999995,"5min":0.69399999999999995,"15min":0.74099999999999999},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46000000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.69999999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70099999999999996,"15min":0.68600000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.36099999999999999},"max":{"1min":0.90300000000000002,"5min":0.92700000000000005,"15min":1.0149999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.66800000000000004},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.376},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.35699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.39000000000000001},"max":{"1min":0.90300000000000002,"5min":0.93700000000000006,"15min":0.997},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.36699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.96099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.40000000000000002},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":1.0349999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:52:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69399999999999995,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.98699999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65500000000000003}]}]},{"osd":2,"up_from":54,"seq":231928234062,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":660,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":675840,"data_stored":323364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5090,"internal_metadata":32369694},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.376,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.223},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.432,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":0.70499999999999996},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.309},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46700000000000003,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":1.006,"15min":1.006},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.67900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79700000000000004,"5min":1.05,"15min":1.05},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":0.93999999999999995},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.96099999999999997,"15min":0.97099999999999997},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":1.006},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95199999999999996,"5min":1.028,"15min":1.028},"last":0.46200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.219},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":1.018},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.88500000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.505},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.373}]},{"osd":3,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.435,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.90800000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.76800000000000002,"5min":0.82299999999999995,"15min":0.83599999999999997},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.41899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":1.0740000000000001},"last":0.622},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.503,"5min":0.42999999999999999,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.96799999999999997},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.1839999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.66000000000000003},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.433},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.70499999999999996,"15min":0.66800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48299999999999998,"15min":0.41099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41099999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.45000000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.224},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575046,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1286144,"data_stored":920047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.83599999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.20300000000000001},"max":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":1.077},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Oct 19 10:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58799999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59299999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.28699999999999998,"15min":0.221},"max":{"1min":0.64500000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.247},"max":{"1min":0.625,"5min":0.74099999999999999,"15min":0.745},"last":0.373}]},{"osd":4,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.47299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.02,"15min":1.1499999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93600000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.99099999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.40200000000000002},"max":{"1min":0.90100000000000002,"5min":0.96299999999999997,"15min":1.1919999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":1.0029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.1910000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.57999999999999996,"5min":0.435,"15min":0.39800000000000002},"max":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":1.0289999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0960000000000001},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:52:53.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-19T10:52:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.399+0000 7f565e307640 1 Processor -- start 2023-10-19T10:52:53.402 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.399+0000 7f565e307640 1 -- start start 2023-10-19T10:52:53.404 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f565e307640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56580b1f30 0x7f56580b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f565e307640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565815d6e0 con 0x7f5658160340 2023-10-19T10:52:53.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f565e307640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56580b2850 con 0x7f565815ddf0 2023-10-19T10:52:53.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f565e307640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565815f030 con 0x7f56580b1f30 2023-10-19T10:52:53.405 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f56577fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5658160340 0x7f56581607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42680/0 (socket says 172.21.15.50:42680) 2023-10-19T10:52:53.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f56577fe640 1 -- 172.21.15.50:0/1583083257 learned_addr learned my addr 172.21.15.50:0/1583083257 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:53.406 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5657fff640 1 --2- 172.21.15.50:0/1583083257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56580b1f30 0x7f56580b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5656ffd640 1 -- 172.21.15.50:0/1583083257 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836845551 0 0) 0x7f565815d6e0 con 0x7f5658160340 2023-10-19T10:52:53.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5656ffd640 1 -- 172.21.15.50:0/1583083257 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5640003520 con 0x7f5658160340 2023-10-19T10:52:53.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5656ffd640 1 -- 172.21.15.50:0/1583083257 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900572553 0 0) 0x7f56580b2850 con 0x7f565815ddf0 2023-10-19T10:52:53.407 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5657fff640 1 -- 172.21.15.50:0/1583083257 >> v1:172.21.15.183:6789/0 conn(0x7f565815ddf0 legacy=0x7f565815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5657fff640 1 -- 172.21.15.50:0/1583083257 >> v1:172.21.15.50:6789/0 conn(0x7f5658160340 legacy=0x7f56581607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.408 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.400+0000 7f5657fff640 1 -- 172.21.15.50:0/1583083257 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565815cdb0 con 0x7f56580b1f30 2023-10-19T10:52:53.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.401+0000 7f5657fff640 1 --2- 172.21.15.50:0/1583083257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56580b1f30 0x7f56580b2310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f564c0097d0 tx=0x7f564c02b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb00596fdbe18b18 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:53.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.401+0000 7f5656ffd640 1 -- 172.21.15.50:0/1583083257 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648001670 con 0x7f56580b1f30 2023-10-19T10:52:53.409 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.401+0000 7f5656ffd640 1 -- 172.21.15.50:0/1583083257 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c02e050 con 0x7f56580b1f30 2023-10-19T10:52:53.410 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.401+0000 7f5656ffd640 1 -- 172.21.15.50:0/1583083257 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c030710 con 0x7f56580b1f30 2023-10-19T10:52:53.414 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 -- 172.21.15.50:0/1583083257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56580b1f30 msgr2=0x7f56580b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 --2- 172.21.15.50:0/1583083257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56580b1f30 0x7f56580b2310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f564c0097d0 tx=0x7f564c02b690 comp rx=0 tx=0).stop 2023-10-19T10:52:53.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 -- 172.21.15.50:0/1583083257 shutdown_connections 2023-10-19T10:52:53.415 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 --2- 172.21.15.50:0/1583083257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56580b1f30 0x7f56580b2310 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 -- 172.21.15.50:0/1583083257 >> 172.21.15.50:0/1583083257 conn(0x7f56580ccf30 msgr2=0x7f56580cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:53.416 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 -- 172.21.15.50:0/1583083257 shutdown_connections 2023-10-19T10:52:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.402+0000 7f565e307640 1 -- 172.21.15.50:0/1583083257 wait complete. 2023-10-19T10:52:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.403+0000 7f565e307640 1 Processor -- start 2023-10-19T10:52:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.403+0000 7f565e307640 1 -- start start 2023-10-19T10:52:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f565e307640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 0x7f56580c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f565e307640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f565815ddf0 0x7f56580c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f565e307640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5658160340 0x7f56580c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f565e307640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580c7720 con 0x7f56580b1f30 2023-10-19T10:52:53.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f565e307640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580c6e00 con 0x7f565815ddf0 2023-10-19T10:52:53.419 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f565e307640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56580c45c0 con 0x7f5658160340 2023-10-19T10:52:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f5657fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 0x7f56580c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f5657fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 0x7f56580c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39120/0 (socket says 172.21.15.50:39120) 2023-10-19T10:52:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.404+0000 7f5657fff640 1 -- 172.21.15.50:0/3530846239 learned_addr learned my addr 172.21.15.50:0/3530846239 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:53.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f565c8a4640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5658160340 0x7f56580c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f56577fe640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f565815ddf0 0x7f56580c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5657fff640 1 -- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5658160340 msgr2=0x7f56580c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5657fff640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5658160340 0x7f56580c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5657fff640 1 -- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f565815ddf0 msgr2=0x7f56580c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5657fff640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f565815ddf0 0x7f56580c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5657fff640 1 -- 172.21.15.50:0/3530846239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c009400 con 0x7f56580b1f30 2023-10-19T10:52:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f565c8a4640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5658160340 0x7f56580c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5657fff640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 0x7f56580c3fe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f564c02b6d0 tx=0x7f564c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0044c0 con 0x7f56580b1f30 2023-10-19T10:52:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c02be90 con 0x7f56580b1f30 2023-10-19T10:52:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56580c48a0 con 0x7f56580b1f30 2023-10-19T10:52:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c030730 con 0x7f56580b1f30 2023-10-19T10:52:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.405+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56580c4e60 con 0x7f56580b1f30 2023-10-19T10:52:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.406+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565815fca0 con 0x7f56580b1f30 2023-10-19T10:52:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.407+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f564c030890 con 0x7f56580b1f30 2023-10-19T10:52:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.407+0000 7f5654ff9640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f564406a050 0x7f564406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.407+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f564c0aa080 con 0x7f56580b1f30 2023-10-19T10:52:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.407+0000 7f56577fe640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f564406a050 0x7f564406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:53.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.408+0000 7f56577fe640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f564406a050 0x7f564406c510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5658072890 tx=0x7f5648001990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:53.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.411+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f564c075870 con 0x7f56580b1f30 2023-10-19T10:52:53.594 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.591+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5658073990 con 0x7f56580b1f30 2023-10-19T10:52:53.595 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.592+0000 7f5654ff9640 1 -- 172.21.15.50:0/3530846239 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+15024 (secure 0 0 0) 0x7f564c04b090 con 0x7f56580b1f30 2023-10-19T10:52:53.595 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:53.596 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":58,"fsid":"2b47c9ed-f967-49ff-9d46-d95ee8c39a83","created":"2023-10-19T10:36:30.337960+0000","modified":"2023-10-19T10:52:49.470583+0000","last_up_change":"2023-10-19T10:46:25.317625+0000","last_in_change":"2023-10-19T10:36:32.547423+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-19T10:36:42.521036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-19T10:36:48.338828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"969faf31-afdd-4ca1-862d-266c0a9d117f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6815","nonce":231212571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6819","nonce":231212571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6823","nonce":231212571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":231212571},{"type":"v1","addr":"172.21.15.50:6821","nonce":231212571}]},"public_addr":"172.21.15.50:6815/231212571","cluster_addr":"172.21.15.50:6819/231212571","heartbeat_back_addr":"172.21.15.50:6823/231212571","heartbeat_front_addr":"172.21.15.50:6821/231212571","state":["exists","up"]},{"osd":1,"uuid":"446ff7aa-e634-46bf-aba0-fa0084bc4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6825","nonce":3823862985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6827","nonce":3823862985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6831","nonce":3823862985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":3823862985},{"type":"v1","addr":"172.21.15.50:6829","nonce":3823862985}]},"public_addr":"172.21.15.50:6825/3823862985","cluster_addr":"172.21.15.50:6827/3823862985","heartbeat_back_addr":"172.21.15.50:6831/3823862985","heartbeat_front_addr":"172.21.15.50:6829/3823862985","state":["exists","up"]},{"osd":2,"uuid":"6852e173-ea11-407a-bc20-c9f6669740c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":54,"up_thru":54,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6806","nonce":323264813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6809","nonce":323264813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6818","nonce":323264813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":323264813},{"type":"v1","addr":"172.21.15.50:6814","nonce":323264813}]},"public_addr":"172.21.15.50:6806/323264813","cluster_addr":"172.21.15.50:6809/323264813","heartbeat_back_addr":"172.21.15.50:6818/323264813","heartbeat_front_addr":"172.21.15.50:6814/323264813","state":["exists","up"]},{"osd":3,"uuid":"b4f1c130-4798-40fa-a122-8b38a2ea0d8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6801","nonce":945885036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6803","nonce":945885036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6812","nonce":945885036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6805","nonce":945885036},{"type":"v1","addr":"172.21.15.50:6807","nonce":945885036}]},"public_addr":"172.21.15.50:6801/945885036","cluster_addr":"172.21.15.50:6803/945885036","heartbeat_back_addr":"172.21.15.50:6812/945885036","heartbeat_front_addr":"172.21.15.50:6807/945885036","state":["exists","up"]},{"osd":4,"uuid":"dbc106ec-a15b-4b9c-82f1-b6e1cefceef0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6801","nonce":2392619067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6803","nonce":2392619067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6807","nonce":2392619067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2392619067},{"type":"v1","addr":"172.21.15.183:6805","nonce":2392619067}]},"public_addr":"172.21.15.183:6801/2392619067","cluster_addr":"172.21.15.183:6803/2392619067","heartbeat_back_addr":"172.21.15.183:6807/2392619067","heartbeat_front_addr":"172.21.15.183:6805/2392619067","state":["exists","up"]},{"osd":5,"uuid":"de88c853-3324-48a2-a105-bf15a3dea21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6809","nonce":3486412186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6811","nonce":3486412186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6815","nonce":3486412186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3486412186},{"type":"v1","addr":"172.21.15.183:6813","nonce":3486412186}]},"public_addr":"172.21.15.183:6809/3486412186","cluster_addr":"172.21.15.183:6811/3486412186","heartbeat_back_addr":"172.21.15.183:6815/3486412186","heartbeat_front_addr":"172.21.15.183:6813/3486412186","state":["exists","up"]},{"osd":6,"uuid":"2acb3a97-1331-427e-a927-b37554c81873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6817","nonce":265753870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6819","nonce":265753870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6823","nonce":265753870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":265753870},{"type":"v1","addr":"172.21.15.183:6821","nonce":265753870}]},"public_addr":"172.21.15.183:6817/265753870","cluster_addr":"172.21.15.183:6819/265753870","heartbeat_back_addr":"172.21.15.183:6823/265753870","heartbeat_front_addr":"172.21.15.183:6821/265753870","state":["exists","up"]},{"osd":7,"uuid":"956a9722-f431-4874-b98b-c3be06fb3e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6825","nonce":3403463202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6827","nonce":3403463202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6831","nonce":3403463202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3403463202},{"type":"v1","addr":"172.21.15.183:6829","nonce":3403463202}]},"public_addr":"172.21.15.183:6825/3403463202","cluster_addr":"172.21.15.183:6827/3403463202","heartbeat_back_addr":"172.21.15.183:6831/3403463202","heartbeat_front_addr":"172.21.15.183:6829/3403463202","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.694539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.690854+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-10-19T10:39:07.171743+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.687633+0000","dead_epoch":51},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.698032+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.757283+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.746922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.701180+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-19T10:36:37.732671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6834/1814941523":"2023-10-20T10:52:46.432892+0000","172.21.15.50:6832/1814941523":"2023-10-20T10:52:46.432892+0000","192.168.0.1:0/216420809":"2023-10-19T11:46:25.724828+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-19T10:52:53.597 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f564406a050 msgr2=0x7f564406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f564406a050 0x7f564406c510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5658072890 tx=0x7f5648001990 comp rx=0 tx=0).stop 2023-10-19T10:52:53.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 msgr2=0x7f56580c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:53.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 0x7f56580c3fe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f564c02b6d0 tx=0x7f564c0046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.598 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 shutdown_connections 2023-10-19T10:52:53.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f564406a050 0x7f564406c510 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5658160340 0x7f56580c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.599 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f565815ddf0 0x7f56580c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 --2- 172.21.15.50:0/3530846239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56580b1f30 0x7f56580c3fe0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:53.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 >> 172.21.15.50:0/3530846239 conn(0x7f56580ccf30 msgr2=0x7f565815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:53.600 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 shutdown_connections 2023-10-19T10:52:53.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:53.594+0000 7f565e307640 1 -- 172.21.15.50:0/3530846239 wait complete. 2023-10-19T10:52:54.609 INFO:tasks.ceph:Scrubbing osd.0 2023-10-19T10:52:54.610 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:54.732 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.729+0000 7f89ae484640 1 Processor -- start 2023-10-19T10:52:54.733 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.730+0000 7f89ae484640 1 -- start start 2023-10-19T10:52:54.735 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.730+0000 7f89ae484640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.730+0000 7f89ae484640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89a815d180 0x7f89a80caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.730+0000 7f89ae484640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a806f270 con 0x7f89a80ccce0 2023-10-19T10:52:54.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.730+0000 7f89ae484640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a815d9c0 con 0x7f89a815d180 2023-10-19T10:52:54.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.730+0000 7f89ae484640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a806f510 con 0x7f89a806fff0 2023-10-19T10:52:54.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89a815d180 0x7f89a80caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.737 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89a815d180 0x7f89a80caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39126/0 (socket says 172.21.15.50:39126) 2023-10-19T10:52:54.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89a7fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f89a80ccce0 0x7f89a80cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42688/0 (socket says 172.21.15.50:42688) 2023-10-19T10:52:54.738 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 -- 172.21.15.50:0/1173042946 learned_addr learned my addr 172.21.15.50:0/1173042946 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:54.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89a77fe640 1 --2- 172.21.15.50:0/1173042946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 -- 172.21.15.50:0/1173042946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 msgr2=0x7f89a815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:54.739 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 --2- 172.21.15.50:0/1173042946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a815cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 -- 172.21.15.50:0/1173042946 >> v1:172.21.15.50:6789/0 conn(0x7f89a80ccce0 legacy=0x7f89a80cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89aca21640 1 -- 172.21.15.50:0/1173042946 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a806f7f0 con 0x7f89a815d180 2023-10-19T10:52:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.731+0000 7f89a6ffd640 1 -- 172.21.15.50:0/1173042946 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160440945 0 0) 0x7f89a806f270 con 0x7f89a80ccce0 2023-10-19T10:52:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89aca21640 1 --2- 172.21.15.50:0/1173042946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a80caa80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009e50 tx=0x7f89a002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1672b1f12eecf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89a6ffd640 1 -- 172.21.15.50:0/1173042946 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a002cb00 con 0x7f89a815d180 2023-10-19T10:52:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89a6ffd640 1 -- 172.21.15.50:0/1173042946 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a002cc60 con 0x7f89a815d180 2023-10-19T10:52:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89a6ffd640 1 -- 172.21.15.50:0/1173042946 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00343f0 con 0x7f89a815d180 2023-10-19T10:52:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 -- 172.21.15.50:0/1173042946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 msgr2=0x7f89a80caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 --2- 172.21.15.50:0/1173042946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a80caa80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009e50 tx=0x7f89a002bf60 comp rx=0 tx=0).stop 2023-10-19T10:52:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 -- 172.21.15.50:0/1173042946 shutdown_connections 2023-10-19T10:52:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 --2- 172.21.15.50:0/1173042946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a80caa80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 --2- 172.21.15.50:0/1173042946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 -- 172.21.15.50:0/1173042946 >> 172.21.15.50:0/1173042946 conn(0x7f89a806dad0 msgr2=0x7f89a806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:54.743 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 -- 172.21.15.50:0/1173042946 shutdown_connections 2023-10-19T10:52:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.732+0000 7f89ae484640 1 -- 172.21.15.50:0/1173042946 wait complete. 2023-10-19T10:52:54.747 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.734+0000 7f89ae484640 1 Processor -- start 2023-10-19T10:52:54.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.734+0000 7f89ae484640 1 -- start start 2023-10-19T10:52:54.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89ae484640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a80c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89ae484640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89a80ccce0 0x7f89a80c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.748 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89ae484640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a81f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89ae484640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c63a0 con 0x7f89a815d180 2023-10-19T10:52:54.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89ae484640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c93d0 con 0x7f89a80ccce0 2023-10-19T10:52:54.749 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89ae484640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81f2130 con 0x7f89a806fff0 2023-10-19T10:52:54.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a81f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a81f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39132/0 (socket says 172.21.15.50:39132) 2023-10-19T10:52:54.750 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 -- 172.21.15.50:0/870451671 learned_addr learned my addr 172.21.15.50:0/870451671 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89a7fff640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a80c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89a77fe640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89a80ccce0 0x7f89a80c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 -- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 msgr2=0x7f89a80c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89a806fff0 0x7f89a80c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 -- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89a80ccce0 msgr2=0x7f89a80c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89a80ccce0 0x7f89a80c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 -- 172.21.15.50:0/870451671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a0009b30 con 0x7f89a815d180 2023-10-19T10:52:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89a77fe640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89a80ccce0 0x7f89a80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89aca21640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a81f1910 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f89a0036040 tx=0x7f89a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0009510 con 0x7f89a815d180 2023-10-19T10:52:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.735+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a0040470 con 0x7f89a815d180 2023-10-19T10:52:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.736+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0040800 con 0x7f89a815d180 2023-10-19T10:52:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.736+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a81f1e50 con 0x7f89a815d180 2023-10-19T10:52:54.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.736+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a80c8b30 con 0x7f89a815d180 2023-10-19T10:52:54.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.736+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89a815d630 con 0x7f89a815d180 2023-10-19T10:52:54.755 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.737+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f89a0040960 con 0x7f89a815d180 2023-10-19T10:52:54.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.737+0000 7f89a4ff9640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f898806a050 0x7f898806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f89a00ab1e0 con 0x7f89a815d180 2023-10-19T10:52:54.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a4ff9640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7f89880709c0 0x7f8988072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.756 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a7fff640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f898806a050 0x7f898806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 --> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89880734d0 con 0x7f89880709c0 2023-10-19T10:52:54.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a00ad7c0 con 0x7f89a815d180 2023-10-19T10:52:54.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a77fe640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7f89880709c0 0x7f8988072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.757 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a77fe640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7f89880709c0 0x7f8988072e00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.738+0000 7f89a7fff640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f898806a050 0x7f898806c510 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8998000f60 tx=0x7f8998007b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.758 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.739+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== osd.0 v2:172.21.15.50:6813/231212571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f89880734d0 con 0x7f89880709c0 2023-10-19T10:52:54.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 --> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f89a806fad0 con 0x7f89880709c0 2023-10-19T10:52:54.759 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89a4ff9640 1 -- 172.21.15.50:0/870451671 <== osd.0 v2:172.21.15.50:6813/231212571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8990009550 con 0x7f89880709c0 2023-10-19T10:52:54.760 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:54.760 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:54.760 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7f89880709c0 msgr2=0x7f8988072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:6813/231212571,v1:172.21.15.50:6815/231212571] conn(0x7f89880709c0 0x7f8988072e00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f898806a050 msgr2=0x7f898806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f898806a050 0x7f898806c510 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8998000f60 tx=0x7f8998007b70 comp rx=0 tx=0).stop 2023-10-19T10:52:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 msgr2=0x7f89a81f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 --2- 172.21.15.50:0/870451671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89a815d180 0x7f89a81f1910 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f89a0036040 tx=0x7f89a0004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89aca21640 1 -- 172.21.15.50:0/870451671 reap_dead start 2023-10-19T10:52:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 shutdown_connections 2023-10-19T10:52:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 >> 172.21.15.50:0/870451671 conn(0x7f89a806dad0 msgr2=0x7f89a806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 shutdown_connections 2023-10-19T10:52:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.752+0000 7f89ae484640 1 -- 172.21.15.50:0/870451671 wait complete. 2023-10-19T10:52:54.766 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-19T10:52:54.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 Processor -- start 2023-10-19T10:52:54.874 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 -- start start 2023-10-19T10:52:54.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 0x7fc1f815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc1f80cd920 0x7fc1f80d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f806aa30 con 0x7fc1f815c570 2023-10-19T10:52:54.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f815ce90 con 0x7fc1f80cd920 2023-10-19T10:52:54.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.871+0000 7fc1fdbd0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f806b2d0 con 0x7fc1f80d0e30 2023-10-19T10:52:54.879 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f77fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc1f815c570 0x7fc1f806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42702/0 (socket says 172.21.15.50:42702) 2023-10-19T10:52:54.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f77fe640 1 -- 172.21.15.50:0/269675331 learned_addr learned my addr 172.21.15.50:0/269675331 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:54.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f6ffd640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 0x7fc1f815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871521301 0 0) 0x7fc1f806aa30 con 0x7fc1f815c570 2023-10-19T10:52:54.880 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f7fff640 1 --2- 172.21.15.50:0/269675331 >> v2:172.21.15.183:3300/0 conn(0x7fc1f80cd920 0x7fc1f80d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e0003520 con 0x7fc1f815c570 2023-10-19T10:52:54.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105374097 0 0) 0x7fc1e0003520 con 0x7fc1f815c570 2023-10-19T10:52:54.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 msgr2=0x7fc1f815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:54.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 0x7fc1f815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 >> v2:172.21.15.183:3300/0 conn(0x7fc1f80cd920 msgr2=0x7fc1f80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:54.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 --2- 172.21.15.50:0/269675331 >> v2:172.21.15.183:3300/0 conn(0x7fc1f80cd920 0x7fc1f80d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80d1270 con 0x7fc1f815c570 2023-10-19T10:52:54.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fc1e8001b70 con 0x7fc1f815c570 2023-10-19T10:52:54.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 >> v1:172.21.15.50:6789/0 conn(0x7fc1f815c570 legacy=0x7fc1f806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 0x7fc1e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1e0004d00 0x7fc1e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f815ce90 con 0x7fc1e0003970 2023-10-19T10:52:54.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f806b2d0 con 0x7fc1e0004d00 2023-10-19T10:52:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e000b980 con 0x7fc1f80d0e30 2023-10-19T10:52:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.872+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f6ffd640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 0x7fc1e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f7fff640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1e0004d00 0x7fc1e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f77fe640 1 -- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 msgr2=0x7fc1e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80d0e30 0x7fc1e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f77fe640 1 -- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1e0004d00 msgr2=0x7fc1e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1e0004d00 0x7fc1e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f77fe640 1 -- 172.21.15.50:0/269675331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0003520 con 0x7fc1e0003970 2023-10-19T10:52:54.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1e0003d90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c530 tx=0x7fc1e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b161960f3601d079 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.888 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8005b20 con 0x7fc1e0003970 2023-10-19T10:52:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.873+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8005c80 con 0x7fc1e0003970 2023-10-19T10:52:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1f67fc640 1 -- 172.21.15.50:0/269675331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8007070 con 0x7fc1e0003970 2023-10-19T10:52:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/269675331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 msgr2=0x7fc1e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/269675331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1e0003d90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c530 tx=0x7fc1e00057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1f7fff640 1 -- 172.21.15.50:0/269675331 reap_dead start 2023-10-19T10:52:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/269675331 shutdown_connections 2023-10-19T10:52:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/269675331 >> 172.21.15.50:0/269675331 conn(0x7fc1f8089ff0 msgr2=0x7fc1f809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/269675331 shutdown_connections 2023-10-19T10:52:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.874+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/269675331 wait complete. 2023-10-19T10:52:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.875+0000 7fc1fdbd0640 1 Processor -- start 2023-10-19T10:52:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.875+0000 7fc1fdbd0640 1 -- start start 2023-10-19T10:52:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1fdbd0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1f814bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1fdbd0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1f814c8c0 0x7fc1f814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1fdbd0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80ce840 0x7fc1f80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1fdbd0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f814c420 con 0x7fc1e0003970 2023-10-19T10:52:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1fdbd0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80cf200 con 0x7fc1f814c8c0 2023-10-19T10:52:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1fdbd0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f814df90 con 0x7fc1f80ce840 2023-10-19T10:52:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1f814bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.896 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1f77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1f814bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39140/0 (socket says 172.21.15.50:39140) 2023-10-19T10:52:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1f77fe640 1 -- 172.21.15.50:0/3073577676 learned_addr learned my addr 172.21.15.50:0/3073577676 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1f7fff640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80ce840 0x7fc1f80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.876+0000 7fc1f6ffd640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1f814c8c0 0x7fc1f814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f77fe640 1 -- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80ce840 msgr2=0x7fc1f80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80ce840 0x7fc1f80cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.898 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f77fe640 1 -- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1f814c8c0 msgr2=0x7fc1f814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1f814c8c0 0x7fc1f814cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f77fe640 1 -- 172.21.15.50:0/3073577676 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f814d380 con 0x7fc1e0003970 2023-10-19T10:52:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f7fff640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80ce840 0x7fc1f80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f6ffd640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1f814c8c0 0x7fc1f814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1f77fe640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1f814bee0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8001270 tx=0x7fc1e80014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e803d070 con 0x7fc1e0003970 2023-10-19T10:52:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e802de20 con 0x7fc1e0003970 2023-10-19T10:52:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f814d660 con 0x7fc1e0003970 2023-10-19T10:52:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80368a0 con 0x7fc1e0003970 2023-10-19T10:52:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.877+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f814dc20 con 0x7fc1e0003970 2023-10-19T10:52:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.878+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f8073990 con 0x7fc1e0003970 2023-10-19T10:52:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.879+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc1e8032040 con 0x7fc1e0003970 2023-10-19T10:52:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.879+0000 7fc1dffff640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc1e406a050 0x7fc1e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.879+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc1e80a9ae0 con 0x7fc1e0003970 2023-10-19T10:52:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.879+0000 7fc1f6ffd640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc1e406a050 0x7fc1e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.880+0000 7fc1f6ffd640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc1e406a050 0x7fc1e406c510 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc1f814fae0 tx=0x7fc1ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:54.883+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc1e8075340 con 0x7fc1e0003970 2023-10-19T10:52:55.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.059+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1f806ac80 con 0x7fc1e406a050 2023-10-19T10:52:55.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.059+0000 7fc1dffff640 1 -- 172.21.15.50:0/3073577676 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc1ec009690 con 0x7fc1e406a050 2023-10-19T10:52:55.063 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 0 to deep-scrub 2023-10-19T10:52:55.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc1e406a050 msgr2=0x7fc1e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc1e406a050 0x7fc1e406c510 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc1f814fae0 tx=0x7fc1ec008040 comp rx=0 tx=0).stop 2023-10-19T10:52:55.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 msgr2=0x7fc1f814bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1f814bee0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8001270 tx=0x7fc1e80014e0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 shutdown_connections 2023-10-19T10:52:55.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc1e406a050 0x7fc1e406c510 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc1f80ce840 0x7fc1f80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1f814c8c0 0x7fc1f814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 --2- 172.21.15.50:0/3073577676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1e0003970 0x7fc1f814bee0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.061+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 >> 172.21.15.50:0/3073577676 conn(0x7fc1f8089ff0 msgr2=0x7fc1f80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.062+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 shutdown_connections 2023-10-19T10:52:55.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.062+0000 7fc1fdbd0640 1 -- 172.21.15.50:0/3073577676 wait complete. 2023-10-19T10:52:55.075 INFO:tasks.ceph:Scrubbing osd.1 2023-10-19T10:52:55.076 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:55.186 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.183+0000 7f0bebeb6640 1 Processor -- start 2023-10-19T10:52:55.187 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.183+0000 7f0bebeb6640 1 -- start start 2023-10-19T10:52:55.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bebeb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be40ccce0 0x7f0be40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.189 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bebeb6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0be415d350 0x7f0be40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bebeb6640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be406f270 con 0x7f0be406fff0 2023-10-19T10:52:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bebeb6640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be40cd600 con 0x7f0be415d350 2023-10-19T10:52:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bebeb6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be415d9c0 con 0x7f0be40ccce0 2023-10-19T10:52:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bea453640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0be415d350 0x7f0be40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bea453640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0be415d350 0x7f0be40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39154/0 (socket says 172.21.15.50:39154) 2023-10-19T10:52:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0bea453640 1 -- 172.21.15.50:0/3843417128 learned_addr learned my addr 172.21.15.50:0/3843417128 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.184+0000 7f0be9c52640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be40ccce0 0x7f0be40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.185+0000 7f0bea453640 1 -- 172.21.15.50:0/3843417128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be40ccce0 msgr2=0x7f0be40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.185+0000 7f0bea453640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be40ccce0 0x7f0be40cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.185+0000 7f0bea453640 1 -- 172.21.15.50:0/3843417128 >> v1:172.21.15.183:6789/0 conn(0x7f0be406fff0 legacy=0x7f0be415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.185+0000 7f0bea453640 1 -- 172.21.15.50:0/3843417128 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be406f510 con 0x7f0be415d350 2023-10-19T10:52:55.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.185+0000 7f0be9c52640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be40ccce0 0x7f0be40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:55.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.185+0000 7f0bea453640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be415d350 0x7f0be40caa80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e50 tx=0x7f0be002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64457190855aff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0be8c50640 1 -- 172.21.15.50:0/3843417128 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be002cb00 con 0x7f0be415d350 2023-10-19T10:52:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0be8c50640 1 -- 172.21.15.50:0/3843417128 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be002cc60 con 0x7f0be415d350 2023-10-19T10:52:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0be8c50640 1 -- 172.21.15.50:0/3843417128 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00343f0 con 0x7f0be415d350 2023-10-19T10:52:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 -- 172.21.15.50:0/3843417128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be415d350 msgr2=0x7f0be40caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be415d350 0x7f0be40caa80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e50 tx=0x7f0be002bf60 comp rx=0 tx=0).stop 2023-10-19T10:52:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 -- 172.21.15.50:0/3843417128 shutdown_connections 2023-10-19T10:52:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be415d350 0x7f0be40caa80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 --2- 172.21.15.50:0/3843417128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be40ccce0 0x7f0be40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 -- 172.21.15.50:0/3843417128 >> 172.21.15.50:0/3843417128 conn(0x7f0be406dad0 msgr2=0x7f0be406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 -- 172.21.15.50:0/3843417128 shutdown_connections 2023-10-19T10:52:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.186+0000 7f0bebeb6640 1 -- 172.21.15.50:0/3843417128 wait complete. 2023-10-19T10:52:55.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.187+0000 7f0bebeb6640 1 Processor -- start 2023-10-19T10:52:55.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.187+0000 7f0bebeb6640 1 -- start start 2023-10-19T10:52:55.200 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0bebeb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be406fff0 0x7f0be40b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0bebeb6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be40ccce0 0x7f0be40c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0bebeb6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be415d350 0x7f0be40c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0bebeb6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40ca8b0 con 0x7f0be40ccce0 2023-10-19T10:52:55.201 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0bebeb6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be414d850 con 0x7f0be415d350 2023-10-19T10:52:55.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0bebeb6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40c9d30 con 0x7f0be406fff0 2023-10-19T10:52:55.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0be9451640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be40ccce0 0x7f0be40c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.202 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0be9451640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be40ccce0 0x7f0be40c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39166/0 (socket says 172.21.15.50:39166) 2023-10-19T10:52:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.188+0000 7f0be9451640 1 -- 172.21.15.50:0/140731695 learned_addr learned my addr 172.21.15.50:0/140731695 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9c52640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be406fff0 0x7f0be40b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bea453640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be415d350 0x7f0be40c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.203 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9451640 1 -- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be406fff0 msgr2=0x7f0be40b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9451640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be406fff0 0x7f0be40b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9451640 1 -- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be415d350 msgr2=0x7f0be40c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9451640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be415d350 0x7f0be40c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.204 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9451640 1 -- 172.21.15.50:0/140731695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0009b30 con 0x7f0be40ccce0 2023-10-19T10:52:55.205 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bea453640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be415d350 0x7f0be40c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:55.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9c52640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0be406fff0 0x7f0be40b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:55.206 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0be9451640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be40ccce0 0x7f0be40c9170 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400c450 tx=0x7f0bd400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40052d0 con 0x7f0be40ccce0 2023-10-19T10:52:55.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd4005430 con 0x7f0be40ccce0 2023-10-19T10:52:55.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be40ca010 con 0x7f0be40ccce0 2023-10-19T10:52:55.207 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4012870 con 0x7f0be40ccce0 2023-10-19T10:52:55.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.189+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be40ca5d0 con 0x7f0be40ccce0 2023-10-19T10:52:55.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.190+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0be406fad0 con 0x7f0be40ccce0 2023-10-19T10:52:55.208 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.190+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0bd40129d0 con 0x7f0be40ccce0 2023-10-19T10:52:55.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0bda7fc640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0bc806a050 0x7f0bc806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0bd4087ec0 con 0x7f0be40ccce0 2023-10-19T10:52:55.209 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0bda7fc640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f0bc80709c0 0x7f0bc8072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 --> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bc80734d0 con 0x7f0bc80709c0 2023-10-19T10:52:55.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd408a580 con 0x7f0be40ccce0 2023-10-19T10:52:55.210 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0be9c52640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0bc806a050 0x7f0bc806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.191+0000 7f0bea453640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f0bc80709c0 0x7f0bc8072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.192+0000 7f0bea453640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f0bc80709c0 0x7f0bc8072e00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.211 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.192+0000 7f0be9c52640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0bc806a050 0x7f0bc806c510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006fd0 tx=0x7f0bdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.192+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== osd.1 v2:172.21.15.50:6824/3823862985 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0bc80734d0 con 0x7f0bc80709c0 2023-10-19T10:52:55.212 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 --> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0be406f7f0 con 0x7f0bc80709c0 2023-10-19T10:52:55.213 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bda7fc640 1 -- 172.21.15.50:0/140731695 <== osd.1 v2:172.21.15.50:6824/3823862985 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0be002cad0 con 0x7f0bc80709c0 2023-10-19T10:52:55.213 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:55.214 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:55.214 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f0bc80709c0 msgr2=0x7f0bc8072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:6824/3823862985,v1:172.21.15.50:6825/3823862985] conn(0x7f0bc80709c0 0x7f0bc8072e00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0bc806a050 msgr2=0x7f0bc806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0bc806a050 0x7f0bc806c510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006fd0 tx=0x7f0bdc008040 comp rx=0 tx=0).stop 2023-10-19T10:52:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be40ccce0 msgr2=0x7f0be40c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bebeb6640 1 --2- 172.21.15.50:0/140731695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0be40ccce0 0x7f0be40c9170 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400c450 tx=0x7f0bd400c910 comp rx=0 tx=0).stop 2023-10-19T10:52:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.205+0000 7f0bea453640 1 -- 172.21.15.50:0/140731695 reap_dead start 2023-10-19T10:52:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.206+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 shutdown_connections 2023-10-19T10:52:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.206+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 >> 172.21.15.50:0/140731695 conn(0x7f0be406dad0 msgr2=0x7f0be415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.206+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 shutdown_connections 2023-10-19T10:52:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.206+0000 7f0bebeb6640 1 -- 172.21.15.50:0/140731695 wait complete. 2023-10-19T10:52:55.220 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-19T10:52:55.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.324+0000 7f1bb4ad9640 1 Processor -- start 2023-10-19T10:52:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.324+0000 7f1bb4ad9640 1 -- start start 2023-10-19T10:52:55.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1bb4ad9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb015a370 0x7f1bb006fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1bb4ad9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1bb00b2390 0x7f1bb015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1bb4ad9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb00caf80 con 0x7f1bb00b1f30 2023-10-19T10:52:55.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1bb4ad9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00d02e0 con 0x7f1bb00b2390 2023-10-19T10:52:55.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1bb4ad9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb015a750 con 0x7f1bb015a370 2023-10-19T10:52:55.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1baed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1bb00b2390 0x7f1bb015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1badd9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1bb00b1f30 0x7f1bb015de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42708/0 (socket says 172.21.15.50:42708) 2023-10-19T10:52:55.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.325+0000 7f1badd9b640 1 -- 172.21.15.50:0/3796622151 learned_addr learned my addr 172.21.15.50:0/3796622151 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:55.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.326+0000 7f1bae59c640 1 --2- 172.21.15.50:0/3796622151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb015a370 0x7f1bb006fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.326+0000 7f1baed9d640 1 -- 172.21.15.50:0/3796622151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb015a370 msgr2=0x7f1bb006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:55.332 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.326+0000 7f1baed9d640 1 --2- 172.21.15.50:0/3796622151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb015a370 0x7f1bb006fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.326+0000 7f1baed9d640 1 -- 172.21.15.50:0/3796622151 >> v1:172.21.15.50:6789/0 conn(0x7f1bb00b1f30 legacy=0x7f1bb015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.326+0000 7f1bad59a640 1 -- 172.21.15.50:0/3796622151 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017657147 0 0) 0x7f1bb00caf80 con 0x7f1bb00b1f30 2023-10-19T10:52:55.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.326+0000 7f1baed9d640 1 -- 172.21.15.50:0/3796622151 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00cf090 con 0x7f1bb00b2390 2023-10-19T10:52:55.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1baed9d640 1 --2- 172.21.15.50:0/3796622151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb00b2390 0x7f1bb015e580 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009d90 tx=0x7f1ba002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aefb9a39584746 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bad59a640 1 -- 172.21.15.50:0/3796622151 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba002cc50 con 0x7f1bb00b2390 2023-10-19T10:52:55.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bad59a640 1 -- 172.21.15.50:0/3796622151 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba002cdb0 con 0x7f1bb00b2390 2023-10-19T10:52:55.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bad59a640 1 -- 172.21.15.50:0/3796622151 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0034660 con 0x7f1bb00b2390 2023-10-19T10:52:55.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/3796622151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb00b2390 msgr2=0x7f1bb015e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/3796622151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb00b2390 0x7f1bb015e580 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009d90 tx=0x7f1ba002bea0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/3796622151 shutdown_connections 2023-10-19T10:52:55.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/3796622151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb00b2390 0x7f1bb015e580 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/3796622151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb015a370 0x7f1bb006fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/3796622151 >> 172.21.15.50:0/3796622151 conn(0x7f1bb00ccf30 msgr2=0x7f1bb00cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/3796622151 shutdown_connections 2023-10-19T10:52:55.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.327+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/3796622151 wait complete. 2023-10-19T10:52:55.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.329+0000 7f1bb4ad9640 1 Processor -- start 2023-10-19T10:52:55.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.329+0000 7f1bb4ad9640 1 -- start start 2023-10-19T10:52:55.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.329+0000 7f1bb4ad9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb00b1f30 0x7f1bb015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.329+0000 7f1bb4ad9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb00b2390 0x7f1bb00c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bb4ad9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 0x7f1bb00c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bb4ad9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00c4b10 con 0x7f1bb015a370 2023-10-19T10:52:55.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bb4ad9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00c4df0 con 0x7f1bb00b2390 2023-10-19T10:52:55.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bb4ad9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb015aa30 con 0x7f1bb00b1f30 2023-10-19T10:52:55.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 0x7f1bb00c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 0x7f1bb00c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39186/0 (socket says 172.21.15.50:39186) 2023-10-19T10:52:55.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 -- 172.21.15.50:0/653954701 learned_addr learned my addr 172.21.15.50:0/653954701 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:55.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bae59c640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb00b1f30 0x7f1bb015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1badd9b640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb00b2390 0x7f1bb00c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 -- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb00b1f30 msgr2=0x7f1bb015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.344 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb00b1f30 0x7f1bb015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 -- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb00b2390 msgr2=0x7f1bb00c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb00b2390 0x7f1bb00c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 -- 172.21.15.50:0/653954701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0009a70 con 0x7f1bb015a370 2023-10-19T10:52:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1badd9b640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb00b2390 0x7f1bb00c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1baed9d640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 0x7f1bb00c44e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009eb0 tx=0x7f1ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba003f420 con 0x7f1bb015a370 2023-10-19T10:52:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba003f580 con 0x7f1bb015a370 2023-10-19T10:52:55.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb015ad10 con 0x7f1bb015a370 2023-10-19T10:52:55.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba003f8f0 con 0x7f1bb015a370 2023-10-19T10:52:55.347 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.330+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb00c5bf0 con 0x7f1bb015a370 2023-10-19T10:52:55.348 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.331+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb015a890 con 0x7f1bb015a370 2023-10-19T10:52:55.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.332+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1ba003fa50 con 0x7f1bb015a370 2023-10-19T10:52:55.349 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.332+0000 7f1ba6ffd640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b9006a050 0x7f1b9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.332+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f1ba00aaf50 con 0x7f1bb015a370 2023-10-19T10:52:55.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.332+0000 7f1bae59c640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b9006a050 0x7f1b9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.350 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.333+0000 7f1bae59c640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b9006a050 0x7f1b9006c510 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80079a0 tx=0x7f1ba8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.352 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.337+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f1bb015a890 con 0x7f1bb015a370 2023-10-19T10:52:55.514 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.511+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bb00d0000 con 0x7f1b9006a050 2023-10-19T10:52:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.511+0000 7f1ba6ffd640 1 -- 172.21.15.50:0/653954701 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1ba80108d0 con 0x7f1b9006a050 2023-10-19T10:52:55.515 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 1 to deep-scrub 2023-10-19T10:52:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b9006a050 msgr2=0x7f1b9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b9006a050 0x7f1b9006c510 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80079a0 tx=0x7f1ba8008040 comp rx=0 tx=0).stop 2023-10-19T10:52:55.517 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 msgr2=0x7f1bb00c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 0x7f1bb00c44e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009eb0 tx=0x7f1ba0004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 shutdown_connections 2023-10-19T10:52:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f1b9006a050 0x7f1b9006c510 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.518 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bb015a370 0x7f1bb00c44e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb00b2390 0x7f1bb00c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.513+0000 7f1bb4ad9640 1 --2- 172.21.15.50:0/653954701 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb00b1f30 0x7f1bb015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.519 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.514+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 >> 172.21.15.50:0/653954701 conn(0x7f1bb00ccf30 msgr2=0x7f1bb00cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.514+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 shutdown_connections 2023-10-19T10:52:55.520 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.514+0000 7f1bb4ad9640 1 -- 172.21.15.50:0/653954701 wait complete. 2023-10-19T10:52:55.527 INFO:tasks.ceph:Scrubbing osd.2 2023-10-19T10:52:55.527 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.635+0000 7f0d23eb1640 1 Processor -- start 2023-10-19T10:52:55.638 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.635+0000 7f0d23eb1640 1 -- start start 2023-10-19T10:52:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d23eb1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0d1c0b55c0 0x7f0d1c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d23eb1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 0x7f0d1c0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.640 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d23eb1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0cfd80 con 0x7f0d1c15c620 2023-10-19T10:52:55.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d23eb1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0ccc10 con 0x7f0d1c0b55c0 2023-10-19T10:52:55.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d23eb1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cda30 con 0x7f0d1c0cf930 2023-10-19T10:52:55.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d2244e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 0x7f0d1c0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.641 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d2144c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0d1c0b55c0 0x7f0d1c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d21c4d640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0d1c15c620 0x7f0d1c0b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:40882/0 (socket says 172.21.15.50:40882) 2023-10-19T10:52:55.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d2244e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 0x7f0d1c0d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.50:34286/0 (socket says 172.21.15.50:34286) 2023-10-19T10:52:55.642 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.636+0000 7f0d21c4d640 1 -- 172.21.15.50:0/3609954001 learned_addr learned my addr 172.21.15.50:0/3609954001 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:55.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d20c4b640 1 -- 172.21.15.50:0/3609954001 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283766434 0 0) 0x7f0d18001110 con 0x7f0d1c15c620 2023-10-19T10:52:55.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d20c4b640 1 -- 172.21.15.50:0/3609954001 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003520 con 0x7f0d1c15c620 2023-10-19T10:52:55.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d2244e640 1 -- 172.21.15.50:0/3609954001 >> v2:172.21.15.183:3300/0 conn(0x7f0d1c0b55c0 msgr2=0x7f0d1c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:55.643 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d2244e640 1 --2- 172.21.15.50:0/3609954001 >> v2:172.21.15.183:3300/0 conn(0x7f0d1c0b55c0 0x7f0d1c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d2244e640 1 -- 172.21.15.50:0/3609954001 >> v1:172.21.15.183:6789/0 conn(0x7f0d1c15c620 legacy=0x7f0d1c0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.644 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d2244e640 1 -- 172.21.15.50:0/3609954001 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c161110 con 0x7f0d1c0cf930 2023-10-19T10:52:55.645 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d2144c640 1 --2- 172.21.15.50:0/3609954001 >> v2:172.21.15.183:3300/0 conn(0x7f0d1c0b55c0 0x7f0d1c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:55.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.637+0000 7f0d2244e640 1 --2- 172.21.15.50:0/3609954001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 0x7f0d1c0d0b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009d90 tx=0x7f0d1402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae75878612557ee0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d20c4b640 1 -- 172.21.15.50:0/3609954001 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1402cc50 con 0x7f0d1c0cf930 2023-10-19T10:52:55.649 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d20c4b640 1 -- 172.21.15.50:0/3609954001 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1402cdb0 con 0x7f0d1c0cf930 2023-10-19T10:52:55.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d20c4b640 1 -- 172.21.15.50:0/3609954001 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14034640 con 0x7f0d1c0cf930 2023-10-19T10:52:55.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 -- 172.21.15.50:0/3609954001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 msgr2=0x7f0d1c0d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.650 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 --2- 172.21.15.50:0/3609954001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 0x7f0d1c0d0b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009d90 tx=0x7f0d1402bea0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 -- 172.21.15.50:0/3609954001 shutdown_connections 2023-10-19T10:52:55.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 --2- 172.21.15.50:0/3609954001 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c0cf930 0x7f0d1c0d0b00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 --2- 172.21.15.50:0/3609954001 >> v2:172.21.15.183:3300/0 conn(0x7f0d1c0b55c0 0x7f0d1c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 -- 172.21.15.50:0/3609954001 >> 172.21.15.50:0/3609954001 conn(0x7f0d1c071110 msgr2=0x7f0d1c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 -- 172.21.15.50:0/3609954001 shutdown_connections 2023-10-19T10:52:55.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.638+0000 7f0d23eb1640 1 -- 172.21.15.50:0/3609954001 wait complete. 2023-10-19T10:52:55.652 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.639+0000 7f0d23eb1640 1 Processor -- start 2023-10-19T10:52:55.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.639+0000 7f0d23eb1640 1 -- start start 2023-10-19T10:52:55.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d23eb1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d1c0b55c0 0x7f0d1c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d23eb1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d1c0cf930 0x7f0d1c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.653 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d23eb1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c15c620 0x7f0d1c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d23eb1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15fb70 con 0x7f0d1c0cf930 2023-10-19T10:52:55.654 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d23eb1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cb220 con 0x7f0d1c0b55c0 2023-10-19T10:52:55.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d23eb1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cb500 con 0x7f0d1c15c620 2023-10-19T10:52:55.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d2144c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d1c0cf930 0x7f0d1c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.655 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d2144c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d1c0cf930 0x7f0d1c152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39190/0 (socket says 172.21.15.50:39190) 2023-10-19T10:52:55.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.640+0000 7f0d2144c640 1 -- 172.21.15.50:0/980068146 learned_addr learned my addr 172.21.15.50:0/980068146 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:55.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2244e640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c15c620 0x7f0d1c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d21c4d640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d1c0b55c0 0x7f0d1c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.656 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2144c640 1 -- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c15c620 msgr2=0x7f0d1c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2144c640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c15c620 0x7f0d1c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2144c640 1 -- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d1c0b55c0 msgr2=0x7f0d1c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.657 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2144c640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d1c0b55c0 0x7f0d1c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2144c640 1 -- 172.21.15.50:0/980068146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c009400 con 0x7f0d1c0cf930 2023-10-19T10:52:55.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2244e640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d1c15c620 0x7f0d1c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:55.658 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d21c4d640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d1c0b55c0 0x7f0d1c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:55.659 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d2144c640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d1c0cf930 0x7f0d1c152930 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00ebe0 tx=0x7f0d0c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c009760 con 0x7f0d1c0cf930 2023-10-19T10:52:55.661 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00cc30 con 0x7f0d1c0cf930 2023-10-19T10:52:55.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00eee0 con 0x7f0d1c0cf930 2023-10-19T10:52:55.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d14009a70 con 0x7f0d1c0cf930 2023-10-19T10:52:55.662 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.641+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c0cbac0 con 0x7f0d1c0cf930 2023-10-19T10:52:55.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.642+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d1c0cd100 con 0x7f0d1c0cf930 2023-10-19T10:52:55.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0d0c013470 con 0x7f0d1c0cf930 2023-10-19T10:52:55.663 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d127fc640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d0006a050 0x7f0d0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0d0c088670 con 0x7f0d1c0cf930 2023-10-19T10:52:55.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d127fc640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] conn(0x7f0d000709c0 0x7f0d00072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 --> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d000734d0 con 0x7f0d000709c0 2023-10-19T10:52:55.664 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d0c08ad30 con 0x7f0d1c0cf930 2023-10-19T10:52:55.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d21c4d640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d0006a050 0x7f0d0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.643+0000 7f0d2244e640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] conn(0x7f0d000709c0 0x7f0d00072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.665 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.644+0000 7f0d2244e640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] conn(0x7f0d000709c0 0x7f0d00072e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.644+0000 7f0d21c4d640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d0006a050 0x7f0d0006c510 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d18007fd0 tx=0x7f0d18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.666 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.644+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== osd.2 v2:172.21.15.50:6804/323264813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0d000734d0 con 0x7f0d000709c0 2023-10-19T10:52:55.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.657+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 --> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0d1c0cce20 con 0x7f0d000709c0 2023-10-19T10:52:55.667 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.657+0000 7f0d127fc640 1 -- 172.21.15.50:0/980068146 <== osd.2 v2:172.21.15.50:6804/323264813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f0d14034540 con 0x7f0d000709c0 2023-10-19T10:52:55.668 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:55.668 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-19T10:52:55.669 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:55.671 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.657+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] conn(0x7f0d000709c0 msgr2=0x7f0d00072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.657+0000 7f0d23eb1640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:6804/323264813,v1:172.21.15.50:6806/323264813] conn(0x7f0d000709c0 0x7f0d00072e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d0006a050 msgr2=0x7f0d0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.672 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f0d0006a050 0x7f0d0006c510 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d18007fd0 tx=0x7f0d18009040 comp rx=0 tx=0).stop 2023-10-19T10:52:55.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d1c0cf930 msgr2=0x7f0d1c152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.673 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 --2- 172.21.15.50:0/980068146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d1c0cf930 0x7f0d1c152930 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00ebe0 tx=0x7f0d0c00c520 comp rx=0 tx=0).stop 2023-10-19T10:52:55.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d2244e640 1 -- 172.21.15.50:0/980068146 reap_dead start 2023-10-19T10:52:55.674 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 shutdown_connections 2023-10-19T10:52:55.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 >> 172.21.15.50:0/980068146 conn(0x7f0d1c071110 msgr2=0x7f0d1c0b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 shutdown_connections 2023-10-19T10:52:55.675 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.658+0000 7f0d23eb1640 1 -- 172.21.15.50:0/980068146 wait complete. 2023-10-19T10:52:55.676 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-19T10:52:55.783 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.780+0000 7fa58a658640 1 Processor -- start 2023-10-19T10:52:55.784 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.781+0000 7fa58a658640 1 -- start start 2023-10-19T10:52:55.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.781+0000 7fa58a658640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa584160340 0x7fa5841607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.781+0000 7fa58a658640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58415d6e0 con 0x7fa58415ddf0 2023-10-19T10:52:55.786 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.781+0000 7fa58a658640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58415f030 con 0x7fa5840b1f30 2023-10-19T10:52:55.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.781+0000 7fa58a658640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58415cdb0 con 0x7fa584160340 2023-10-19T10:52:55.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.781+0000 7fa588bf5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa58415ddf0 0x7fa58415e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42710/0 (socket says 172.21.15.50:42710) 2023-10-19T10:52:55.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa588bf5640 1 -- 172.21.15.50:0/4181174115 learned_addr learned my addr 172.21.15.50:0/4181174115 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:55.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa5837fe640 1 --2- 172.21.15.50:0/4181174115 >> v2:172.21.15.183:3300/0 conn(0x7fa584160340 0x7fa5841607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302912279 0 0) 0x7fa58415d6e0 con 0x7fa58415ddf0 2023-10-19T10:52:55.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa56c003520 con 0x7fa58415ddf0 2023-10-19T10:52:55.788 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677653983 0 0) 0x7fa58415f030 con 0x7fa5840b1f30 2023-10-19T10:52:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa58415d6e0 con 0x7fa5840b1f30 2023-10-19T10:52:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa5837fe640 1 -- 172.21.15.50:0/4181174115 >> v1:172.21.15.183:6789/0 conn(0x7fa5840b1f30 legacy=0x7fa5840b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa5837fe640 1 -- 172.21.15.50:0/4181174115 >> v1:172.21.15.50:6789/0 conn(0x7fa58415ddf0 legacy=0x7fa58415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.782+0000 7fa5837fe640 1 -- 172.21.15.50:0/4181174115 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58415d090 con 0x7fa584160340 2023-10-19T10:52:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.783+0000 7fa5837fe640 1 --2- 172.21.15.50:0/4181174115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa584160340 0x7fa5841607a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa5780097a0 tx=0x7fa57802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76857e249224e5de server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.783+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578031830 con 0x7fa584160340 2023-10-19T10:52:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.783+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa578031990 con 0x7fa584160340 2023-10-19T10:52:55.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.783+0000 7fa582ffd640 1 -- 172.21.15.50:0/4181174115 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578031d20 con 0x7fa584160340 2023-10-19T10:52:55.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 -- 172.21.15.50:0/4181174115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa584160340 msgr2=0x7fa5841607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 --2- 172.21.15.50:0/4181174115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa584160340 0x7fa5841607a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa5780097a0 tx=0x7fa57802b8d0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 -- 172.21.15.50:0/4181174115 shutdown_connections 2023-10-19T10:52:55.794 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 --2- 172.21.15.50:0/4181174115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa584160340 0x7fa5841607a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 -- 172.21.15.50:0/4181174115 >> 172.21.15.50:0/4181174115 conn(0x7fa5840ccf30 msgr2=0x7fa5840cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 -- 172.21.15.50:0/4181174115 shutdown_connections 2023-10-19T10:52:55.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.784+0000 7fa58a658640 1 -- 172.21.15.50:0/4181174115 wait complete. 2023-10-19T10:52:55.795 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.785+0000 7fa58a658640 1 Processor -- start 2023-10-19T10:52:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.785+0000 7fa58a658640 1 -- start start 2023-10-19T10:52:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa58a658640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5840b1f30 0x7fa5840b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa58a658640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa58415ddf0 0x7fa5840b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa58a658640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 0x7fa5840d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa58a658640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b4ff0 con 0x7fa584160340 2023-10-19T10:52:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa58a658640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58414a800 con 0x7fa58415ddf0 2023-10-19T10:52:55.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa58a658640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58414a520 con 0x7fa5840b1f30 2023-10-19T10:52:55.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 0x7fa5840d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 0x7fa5840d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39202/0 (socket says 172.21.15.50:39202) 2023-10-19T10:52:55.798 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 -- 172.21.15.50:0/61999274 learned_addr learned my addr 172.21.15.50:0/61999274 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa583fff640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5840b1f30 0x7fa5840b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa5837fe640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa58415ddf0 0x7fa5840b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 -- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5840b1f30 msgr2=0x7fa5840b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5840b1f30 0x7fa5840b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 -- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa58415ddf0 msgr2=0x7fa5840b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa58415ddf0 0x7fa5840b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.786+0000 7fa588bf5640 1 -- 172.21.15.50:0/61999274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa578009400 con 0x7fa584160340 2023-10-19T10:52:55.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.787+0000 7fa588bf5640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 0x7fa5840d0890 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00cba0 tx=0x7fa57c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.787+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00ace0 con 0x7fa584160340 2023-10-19T10:52:55.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.787+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c00ae40 con 0x7fa584160340 2023-10-19T10:52:55.801 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.787+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5840b5220 con 0x7fa584160340 2023-10-19T10:52:55.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.787+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c012620 con 0x7fa584160340 2023-10-19T10:52:55.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.787+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5840b57e0 con 0x7fa584160340 2023-10-19T10:52:55.802 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.788+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5840b6240 con 0x7fa584160340 2023-10-19T10:52:55.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.788+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa57c007070 con 0x7fa584160340 2023-10-19T10:52:55.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.789+0000 7fa580ff9640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa57006a050 0x7fa57006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:55.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.789+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fa57c0883a0 con 0x7fa584160340 2023-10-19T10:52:55.803 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.789+0000 7fa583fff640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa57006a050 0x7fa57006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:55.804 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.789+0000 7fa583fff640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa57006a050 0x7fa57006c510 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa5740011e0 tx=0x7fa574009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:55.806 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.793+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fa57c053cd0 con 0x7fa584160340 2023-10-19T10:52:55.970 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.967+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa58415fd20 con 0x7fa57006a050 2023-10-19T10:52:55.971 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.968+0000 7fa580ff9640 1 -- 172.21.15.50:0/61999274 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa574001430 con 0x7fa57006a050 2023-10-19T10:52:55.971 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 2 to deep-scrub 2023-10-19T10:52:55.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa57006a050 msgr2=0x7fa57006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa57006a050 0x7fa57006c510 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa5740011e0 tx=0x7fa574009040 comp rx=0 tx=0).stop 2023-10-19T10:52:55.973 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 msgr2=0x7fa5840d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:55.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 0x7fa5840d0890 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00cba0 tx=0x7fa57c00a430 comp rx=0 tx=0).stop 2023-10-19T10:52:55.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 shutdown_connections 2023-10-19T10:52:55.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fa57006a050 0x7fa57006c510 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.974 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa584160340 0x7fa5840d0890 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa58415ddf0 0x7fa5840b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 --2- 172.21.15.50:0/61999274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa5840b1f30 0x7fa5840b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:55.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 >> 172.21.15.50:0/61999274 conn(0x7fa5840ccf30 msgr2=0x7fa5840b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:55.975 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 shutdown_connections 2023-10-19T10:52:55.976 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:55.970+0000 7fa58a658640 1 -- 172.21.15.50:0/61999274 wait complete. 2023-10-19T10:52:55.983 INFO:tasks.ceph:Scrubbing osd.3 2023-10-19T10:52:55.984 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:56.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 Processor -- start 2023-10-19T10:52:56.094 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 -- start start 2023-10-19T10:52:56.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd03c15c620 0x7fd03c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd03c0b44f0 0x7fd03c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.097 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0b4980 con 0x7fd03c15c620 2023-10-19T10:52:56.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c06a860 con 0x7fd03c0b44f0 2023-10-19T10:52:56.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.091+0000 7fd040bce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c06ab40 con 0x7fd03c0ce950 2023-10-19T10:52:56.098 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd03c15c620 0x7fd03c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd03c15c620 0x7fd03c0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39214/0 (socket says 172.21.15.50:39214) 2023-10-19T10:52:56.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 -- 172.21.15.50:0/4176140640 learned_addr learned my addr 172.21.15.50:0/4176140640 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd031d9b640 1 --2- 172.21.15.50:0/4176140640 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.099 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03ad9d640 1 --2- 172.21.15.50:0/4176140640 >> v2:172.21.15.183:3300/0 conn(0x7fd03c0b44f0 0x7fd03c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 -- 172.21.15.50:0/4176140640 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 msgr2=0x7fd03c0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 --2- 172.21.15.50:0/4176140640 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c0b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 -- 172.21.15.50:0/4176140640 >> v2:172.21.15.183:3300/0 conn(0x7fd03c0b44f0 msgr2=0x7fd03c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.100 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 --2- 172.21.15.50:0/4176140640 >> v2:172.21.15.183:3300/0 conn(0x7fd03c0b44f0 0x7fd03c0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.092+0000 7fd03a59c640 1 -- 172.21.15.50:0/4176140640 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c06ae20 con 0x7fd03c15c620 2023-10-19T10:52:56.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd03a59c640 1 --2- 172.21.15.50:0/4176140640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0ce410 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd03c06b870 tx=0x7fd02c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b99d9d0b4f15491 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.101 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd039d9b640 1 -- 172.21.15.50:0/4176140640 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c030860 con 0x7fd03c15c620 2023-10-19T10:52:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd039d9b640 1 -- 172.21.15.50:0/4176140640 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c0309c0 con 0x7fd03c15c620 2023-10-19T10:52:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd039d9b640 1 -- 172.21.15.50:0/4176140640 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c030d50 con 0x7fd03c15c620 2023-10-19T10:52:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 -- 172.21.15.50:0/4176140640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 msgr2=0x7fd03c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.102 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 --2- 172.21.15.50:0/4176140640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0ce410 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd03c06b870 tx=0x7fd02c02b9a0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 -- 172.21.15.50:0/4176140640 shutdown_connections 2023-10-19T10:52:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 --2- 172.21.15.50:0/4176140640 >> v2:172.21.15.183:3300/0 conn(0x7fd03c0b44f0 0x7fd03c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 --2- 172.21.15.50:0/4176140640 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.103 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 --2- 172.21.15.50:0/4176140640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0ce410 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 -- 172.21.15.50:0/4176140640 >> 172.21.15.50:0/4176140640 conn(0x7fd03c071320 msgr2=0x7fd03c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.093+0000 7fd040bce640 1 -- 172.21.15.50:0/4176140640 shutdown_connections 2023-10-19T10:52:56.104 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.094+0000 7fd040bce640 1 -- 172.21.15.50:0/4176140640 wait complete. 2023-10-19T10:52:56.107 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.095+0000 7fd040bce640 1 Processor -- start 2023-10-19T10:52:56.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.095+0000 7fd040bce640 1 -- start start 2023-10-19T10:52:56.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd040bce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c0b44f0 0x7fd03c0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.108 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd040bce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c1f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd040bce640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd040bce640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0ca9a0 con 0x7fd03c15c620 2023-10-19T10:52:56.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd040bce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0abd60 con 0x7fd03c0b44f0 2023-10-19T10:52:56.109 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd040bce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0ca650 con 0x7fd03c0ce950 2023-10-19T10:52:56.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39216/0 (socket says 172.21.15.50:39216) 2023-10-19T10:52:56.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 -- 172.21.15.50:0/1474074360 learned_addr learned my addr 172.21.15.50:0/1474074360 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.110 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd031d9b640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c1f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03a59c640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c0b44f0 0x7fd03c0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 -- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 msgr2=0x7fd03c1f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.111 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c1f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 -- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c0b44f0 msgr2=0x7fd03c0b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c0b44f0 0x7fd03c0b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 -- 172.21.15.50:0/1474074360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0094d0 con 0x7fd03c15c620 2023-10-19T10:52:56.112 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd031d9b640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd03c0ce950 0x7fd03c1f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:56.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd03ad9d640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0cadc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd034007e60 tx=0x7fd03400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.096+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034014620 con 0x7fd03c15c620 2023-10-19T10:52:56.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.097+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03400ee90 con 0x7fd03c15c620 2023-10-19T10:52:56.113 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.097+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0340149e0 con 0x7fd03c15c620 2023-10-19T10:52:56.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.097+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c1f2130 con 0x7fd03c15c620 2023-10-19T10:52:56.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.097+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c0b0d50 con 0x7fd03c15c620 2023-10-19T10:52:56.114 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.097+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd03c06b3e0 con 0x7fd03c15c620 2023-10-19T10:52:56.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.098+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd034014b40 con 0x7fd03c15c620 2023-10-19T10:52:56.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.098+0000 7fd0337fe640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd01c06a050 0x7fd01c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.098+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fd034088c80 con 0x7fd03c15c620 2023-10-19T10:52:56.115 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd0337fe640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fd01c0709c0 0x7fd01c072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd03a59c640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd01c06a050 0x7fd01c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 --> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd01c0734d0 con 0x7fd01c0709c0 2023-10-19T10:52:56.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd03408b340 con 0x7fd03c15c620 2023-10-19T10:52:56.116 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd031d9b640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fd01c0709c0 0x7fd01c072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd031d9b640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fd01c0709c0 0x7fd01c072e00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.099+0000 7fd03a59c640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd01c06a050 0x7fd01c06c510 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd02c036000 tx=0x7fd02c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.117 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.100+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== osd.3 v2:172.21.15.50:6800/945885036 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd01c0734d0 con 0x7fd01c0709c0 2023-10-19T10:52:56.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 --> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd03c06b100 con 0x7fd01c0709c0 2023-10-19T10:52:56.118 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd0337fe640 1 -- 172.21.15.50:0/1474074360 <== osd.3 v2:172.21.15.50:6800/945885036 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd024009550 con 0x7fd01c0709c0 2023-10-19T10:52:56.119 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:56.119 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:56.120 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:56.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fd01c0709c0 msgr2=0x7fd01c072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:6800/945885036,v1:172.21.15.50:6801/945885036] conn(0x7fd01c0709c0 0x7fd01c072e00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.121 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd01c06a050 msgr2=0x7fd01c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fd01c06a050 0x7fd01c06c510 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd02c036000 tx=0x7fd02c03b040 comp rx=0 tx=0).stop 2023-10-19T10:52:56.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 msgr2=0x7fd03c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 --2- 172.21.15.50:0/1474074360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd03c15c620 0x7fd03c0cadc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd034007e60 tx=0x7fd03400e4b0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.122 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd03ad9d640 1 -- 172.21.15.50:0/1474074360 reap_dead start 2023-10-19T10:52:56.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 shutdown_connections 2023-10-19T10:52:56.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 >> 172.21.15.50:0/1474074360 conn(0x7fd03c071320 msgr2=0x7fd03c0b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.123 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 shutdown_connections 2023-10-19T10:52:56.124 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.113+0000 7fd040bce640 1 -- 172.21.15.50:0/1474074360 wait complete. 2023-10-19T10:52:56.126 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-19T10:52:56.233 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.230+0000 7f06d5b2e640 1 Processor -- start 2023-10-19T10:52:56.234 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.230+0000 7f06d5b2e640 1 -- start start 2023-10-19T10:52:56.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06d5b2e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 0x7f06d00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.237 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06d5b2e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d0069800 con 0x7f06d00ccc20 2023-10-19T10:52:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06d5b2e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d00cdc30 con 0x7f06d015da60 2023-10-19T10:52:56.238 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06d5b2e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00cdf10 con 0x7f06d00cd730 2023-10-19T10:52:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06ceffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f06d00ccc20 0x7f06d0069c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42712/0 (socket says 172.21.15.50:42712) 2023-10-19T10:52:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06ceffd640 1 -- 172.21.15.50:0/1874152146 learned_addr learned my addr 172.21.15.50:0/1874152146 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:56.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935675349 0 0) 0x7f06d0069800 con 0x7f06d00ccc20 2023-10-19T10:52:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b8003520 con 0x7f06d00ccc20 2023-10-19T10:52:56.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.231+0000 7f06cffff640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 0x7f06d00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2384234522 0 0) 0x7f06b8003520 con 0x7f06d00ccc20 2023-10-19T10:52:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 msgr2=0x7f06d00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.241 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 0x7f06d00cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 >> v1:172.21.15.183:6789/0 conn(0x7f06d015da60 legacy=0x7f06d00cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00ce1f0 con 0x7f06d00ccc20 2023-10-19T10:52:56.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f06c0001670 con 0x7f06d00ccc20 2023-10-19T10:52:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 >> v1:172.21.15.50:6789/0 conn(0x7f06d00ccc20 legacy=0x7f06d0069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 0x7f06b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06b8003f20 0x7f06b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06b8004d00 0x7f06b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00cdf10 con 0x7f06b8004d00 2023-10-19T10:52:56.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b800bb10 con 0x7f06b8003f20 2023-10-19T10:52:56.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b800bcd0 con 0x7f06d00cd730 2023-10-19T10:52:56.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cffff640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06b8004d00 0x7f06b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cf7fe640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 0x7f06b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06ceffd640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06b8003f20 0x7f06b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.246 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cffff640 1 -- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 msgr2=0x7f06b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cffff640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06d00cd730 0x7f06b80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cffff640 1 -- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06b8003f20 msgr2=0x7f06b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.247 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cffff640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06b8003f20 0x7f06b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.232+0000 7f06cffff640 1 -- 172.21.15.50:0/1874152146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8003520 con 0x7f06b8004d00 2023-10-19T10:52:56.248 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.233+0000 7f06cffff640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06b8004d00 0x7f06b8005180 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009f90 tx=0x7f06b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38b2af67d3943b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.233+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c802cbd0 con 0x7f06b8004d00 2023-10-19T10:52:56.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.233+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c800f040 con 0x7f06b8004d00 2023-10-19T10:52:56.249 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.233+0000 7f06ce7fc640 1 -- 172.21.15.50:0/1874152146 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8034440 con 0x7f06b8004d00 2023-10-19T10:52:56.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06d5b2e640 1 -- 172.21.15.50:0/1874152146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06b8004d00 msgr2=0x7f06b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/1874152146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06b8004d00 0x7f06b8005180 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009f90 tx=0x7f06b80057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.250 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06cffff640 1 -- 172.21.15.50:0/1874152146 reap_dead start 2023-10-19T10:52:56.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06d5b2e640 1 -- 172.21.15.50:0/1874152146 shutdown_connections 2023-10-19T10:52:56.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06d5b2e640 1 -- 172.21.15.50:0/1874152146 >> 172.21.15.50:0/1874152146 conn(0x7f06d00cb3c0 msgr2=0x7f06d00cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.251 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06d5b2e640 1 -- 172.21.15.50:0/1874152146 shutdown_connections 2023-10-19T10:52:56.252 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.234+0000 7f06d5b2e640 1 -- 172.21.15.50:0/1874152146 wait complete. 2023-10-19T10:52:56.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.235+0000 7f06d5b2e640 1 Processor -- start 2023-10-19T10:52:56.255 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.235+0000 7f06d5b2e640 1 -- start start 2023-10-19T10:52:56.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06d5b2e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06b8004d00 0x7f06d00c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06d5b2e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 0x7f06d00c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.256 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06d5b2e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d00c2f80 0x7f06d00c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06d5b2e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00c91d0 con 0x7f06d00c89c0 2023-10-19T10:52:56.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06d5b2e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00c3940 con 0x7f06d00c2f80 2023-10-19T10:52:56.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06d5b2e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0158330 con 0x7f06b8004d00 2023-10-19T10:52:56.257 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 0x7f06d00c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 0x7f06d00c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39230/0 (socket says 172.21.15.50:39230) 2023-10-19T10:52:56.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 -- 172.21.15.50:0/3985229527 learned_addr learned my addr 172.21.15.50:0/3985229527 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.258 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06cf7fe640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06b8004d00 0x7f06d00c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06cffff640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d00c2f80 0x7f06d00c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 -- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06b8004d00 msgr2=0x7f06d00c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06b8004d00 0x7f06d00c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 -- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d00c2f80 msgr2=0x7f06d00c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d00c2f80 0x7f06d00c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06ceffd640 1 -- 172.21.15.50:0/3985229527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0157570 con 0x7f06d00c89c0 2023-10-19T10:52:56.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.236+0000 7f06cf7fe640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06b8004d00 0x7f06d00c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:56.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.237+0000 7f06ceffd640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 0x7f06d00c7f60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f06c00072c0 tx=0x7f06c0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.237+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000f960 con 0x7f06d00c89c0 2023-10-19T10:52:56.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.237+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c000fae0 con 0x7f06d00c89c0 2023-10-19T10:52:56.261 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.237+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d0157850 con 0x7f06d00c89c0 2023-10-19T10:52:56.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.237+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000fe90 con 0x7f06d00c89c0 2023-10-19T10:52:56.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.237+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d0157e10 con 0x7f06d00c89c0 2023-10-19T10:52:56.262 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.238+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d0073990 con 0x7f06d00c89c0 2023-10-19T10:52:56.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.238+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f06c001d460 con 0x7f06d00c89c0 2023-10-19T10:52:56.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.239+0000 7f06b7fff640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f06bc06a050 0x7f06bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.239+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f06c0088580 con 0x7f06d00c89c0 2023-10-19T10:52:56.263 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.239+0000 7f06cf7fe640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f06bc06a050 0x7f06bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.239+0000 7f06cf7fe640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f06bc06a050 0x7f06bc06c510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f06c40036b0 tx=0x7f06c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.264 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.243+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f06c0053e00 con 0x7f06d00c89c0 2023-10-19T10:52:56.420 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.418+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f06d00ae190 con 0x7f06bc06a050 2023-10-19T10:52:56.421 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.418+0000 7f06b7fff640 1 -- 172.21.15.50:0/3985229527 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f06c40015e0 con 0x7f06bc06a050 2023-10-19T10:52:56.422 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 3 to deep-scrub 2023-10-19T10:52:56.423 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f06bc06a050 msgr2=0x7f06bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f06bc06a050 0x7f06bc06c510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f06c40036b0 tx=0x7f06c4009040 comp rx=0 tx=0).stop 2023-10-19T10:52:56.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 msgr2=0x7f06d00c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.424 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 0x7f06d00c7f60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f06c00072c0 tx=0x7f06c0007780 comp rx=0 tx=0).stop 2023-10-19T10:52:56.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 shutdown_connections 2023-10-19T10:52:56.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f06bc06a050 0x7f06bc06c510 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.425 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d00c2f80 0x7f06d00c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06d00c89c0 0x7f06d00c7f60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 --2- 172.21.15.50:0/3985229527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06b8004d00 0x7f06d00c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.426 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 >> 172.21.15.50:0/3985229527 conn(0x7f06d00cb3c0 msgr2=0x7f06d00b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 shutdown_connections 2023-10-19T10:52:56.427 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.420+0000 7f06d5b2e640 1 -- 172.21.15.50:0/3985229527 wait complete. 2023-10-19T10:52:56.434 INFO:tasks.ceph:Scrubbing osd.4 2023-10-19T10:52:56.434 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:56.545 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.542+0000 7f9345bde640 1 Processor -- start 2023-10-19T10:52:56.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.542+0000 7f9345bde640 1 -- start start 2023-10-19T10:52:56.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f9345bde640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f934015c620 0x7f93400c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f9345bde640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f934006e5f0 0x7f93400ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.548 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f9345bde640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9340070990 con 0x7f93400cd1c0 2023-10-19T10:52:56.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f9345bde640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934006a340 con 0x7f934006e5f0 2023-10-19T10:52:56.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f9345bde640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400c9c40 con 0x7f934015c620 2023-10-19T10:52:56.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f934006e5f0 0x7f93400ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f934006e5f0 0x7f93400ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39246/0 (socket says 172.21.15.50:39246) 2023-10-19T10:52:56.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 -- 172.21.15.50:0/940530307 learned_addr learned my addr 172.21.15.50:0/940530307 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f93400cd1c0 0x7f9340070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42722/0 (socket says 172.21.15.50:42722) 2023-10-19T10:52:56.550 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933f7fe640 1 --2- 172.21.15.50:0/940530307 >> v2:172.21.15.183:3300/0 conn(0x7f934015c620 0x7f93400c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 -- 172.21.15.50:0/940530307 >> v2:172.21.15.183:3300/0 conn(0x7f934015c620 msgr2=0x7f93400c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 --2- 172.21.15.50:0/940530307 >> v2:172.21.15.183:3300/0 conn(0x7f934015c620 0x7f93400c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 -- 172.21.15.50:0/940530307 >> v1:172.21.15.50:6789/0 conn(0x7f93400cd1c0 legacy=0x7f9340070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.551 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933effd640 1 -- 172.21.15.50:0/940530307 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400cd5d0 con 0x7f934006e5f0 2023-10-19T10:52:56.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933e7fc640 1 -- 172.21.15.50:0/940530307 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101431249 0 0) 0x7f9340070990 con 0x7f93400cd1c0 2023-10-19T10:52:56.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.543+0000 7f933f7fe640 1 --2- 172.21.15.50:0/940530307 >> v2:172.21.15.183:3300/0 conn(0x7f934015c620 0x7f93400c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:56.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.544+0000 7f933effd640 1 --2- 172.21.15.50:0/940530307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934006e5f0 0x7f93400ccc80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f93400b5220 tx=0x7f932802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79a39a87e19b452 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.544+0000 7f933e7fc640 1 -- 172.21.15.50:0/940530307 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328031870 con 0x7f934006e5f0 2023-10-19T10:52:56.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.544+0000 7f933e7fc640 1 -- 172.21.15.50:0/940530307 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93280319d0 con 0x7f934006e5f0 2023-10-19T10:52:56.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.544+0000 7f933e7fc640 1 -- 172.21.15.50:0/940530307 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328031d60 con 0x7f934006e5f0 2023-10-19T10:52:56.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 -- 172.21.15.50:0/940530307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934006e5f0 msgr2=0x7f93400ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 --2- 172.21.15.50:0/940530307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934006e5f0 0x7f93400ccc80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f93400b5220 tx=0x7f932802b910 comp rx=0 tx=0).stop 2023-10-19T10:52:56.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 -- 172.21.15.50:0/940530307 shutdown_connections 2023-10-19T10:52:56.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 --2- 172.21.15.50:0/940530307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934006e5f0 0x7f93400ccc80 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 --2- 172.21.15.50:0/940530307 >> v2:172.21.15.183:3300/0 conn(0x7f934015c620 0x7f93400c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 -- 172.21.15.50:0/940530307 >> 172.21.15.50:0/940530307 conn(0x7f93400cbb30 msgr2=0x7f93400cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 -- 172.21.15.50:0/940530307 shutdown_connections 2023-10-19T10:52:56.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.545+0000 7f9345bde640 1 -- 172.21.15.50:0/940530307 wait complete. 2023-10-19T10:52:56.558 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.546+0000 7f9345bde640 1 Processor -- start 2023-10-19T10:52:56.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.546+0000 7f9345bde640 1 -- start start 2023-10-19T10:52:56.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f9345bde640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f934006e5f0 0x7f93401f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f9345bde640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93400cd1c0 0x7f93400b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.559 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f9345bde640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934015c620 0x7f93400b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f9345bde640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401f2d80 con 0x7f934015c620 2023-10-19T10:52:56.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f9345bde640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93400ac610 con 0x7f934006e5f0 2023-10-19T10:52:56.560 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f9345bde640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93400ac8f0 con 0x7f93400cd1c0 2023-10-19T10:52:56.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934015c620 0x7f93400b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934015c620 0x7f93400b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39256/0 (socket says 172.21.15.50:39256) 2023-10-19T10:52:56.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 -- 172.21.15.50:0/260482435 learned_addr learned my addr 172.21.15.50:0/260482435 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.561 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933effd640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93400cd1c0 0x7f93400b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933f7fe640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f934006e5f0 0x7f93401f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 -- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93400cd1c0 msgr2=0x7f93400b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.562 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93400cd1c0 0x7f93400b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 -- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f934006e5f0 msgr2=0x7f93401f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f934006e5f0 0x7f93401f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.547+0000 7f933ffff640 1 -- 172.21.15.50:0/260482435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280094d0 con 0x7f934015c620 2023-10-19T10:52:56.563 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.548+0000 7f933ffff640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934015c620 0x7f93400b4920 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f933800c7a0 tx=0x7f933800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.548+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800a640 con 0x7f934015c620 2023-10-19T10:52:56.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.548+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933800a7a0 con 0x7f934015c620 2023-10-19T10:52:56.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.548+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93400acbd0 con 0x7f934015c620 2023-10-19T10:52:56.564 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.548+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800abd0 con 0x7f934015c620 2023-10-19T10:52:56.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.548+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93400ad190 con 0x7f934015c620 2023-10-19T10:52:56.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.549+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9340069aa0 con 0x7f934015c620 2023-10-19T10:52:56.565 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.549+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f933800ad30 con 0x7f934015c620 2023-10-19T10:52:56.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f9337fff640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f932006a050 0x7f932006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f9338087bf0 con 0x7f934015c620 2023-10-19T10:52:56.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f9337fff640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7f93200709c0 0x7f9320072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.566 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 --> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93200734d0 con 0x7f93200709c0 2023-10-19T10:52:56.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933808a2b0 con 0x7f934015c620 2023-10-19T10:52:56.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f933f7fe640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f932006a050 0x7f932006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.567 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f933effd640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7f93200709c0 0x7f9320072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f933f7fe640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f932006a050 0x7f932006c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9330006fd0 tx=0x7f9330008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.550+0000 7f933effd640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7f93200709c0 0x7f9320072e00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.568 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.551+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== osd.4 v2:172.21.15.183:6800/2392619067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f93200734d0 con 0x7f93200709c0 2023-10-19T10:52:56.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.564+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 --> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9340069800 con 0x7f93200709c0 2023-10-19T10:52:56.569 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9337fff640 1 -- 172.21.15.50:0/260482435 <== osd.4 v2:172.21.15.183:6800/2392619067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9328031b40 con 0x7f93200709c0 2023-10-19T10:52:56.570 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:56.570 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:56.570 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:56.571 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7f93200709c0 msgr2=0x7f9320072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6800/2392619067,v1:172.21.15.183:6801/2392619067] conn(0x7f93200709c0 0x7f9320072e00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f932006a050 msgr2=0x7f932006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f932006a050 0x7f932006c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9330006fd0 tx=0x7f9330008040 comp rx=0 tx=0).stop 2023-10-19T10:52:56.572 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934015c620 msgr2=0x7f93400b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 --2- 172.21.15.50:0/260482435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f934015c620 0x7f93400b4920 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f933800c7a0 tx=0x7f933800cc60 comp rx=0 tx=0).stop 2023-10-19T10:52:56.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f933ffff640 1 -- 172.21.15.50:0/260482435 reap_dead start 2023-10-19T10:52:56.573 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 shutdown_connections 2023-10-19T10:52:56.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 >> 172.21.15.50:0/260482435 conn(0x7f93400cbb30 msgr2=0x7f93401e2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 shutdown_connections 2023-10-19T10:52:56.574 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.565+0000 7f9345bde640 1 -- 172.21.15.50:0/260482435 wait complete. 2023-10-19T10:52:56.578 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-19T10:52:56.686 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 Processor -- start 2023-10-19T10:52:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 -- start start 2023-10-19T10:52:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f05240b1f30 0x7f05240b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0524160340 0x7f05241607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052415ee00 con 0x7f052406f270 2023-10-19T10:52:56.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052415d6e0 con 0x7f05240b1f30 2023-10-19T10:52:56.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.683+0000 7f0528ee9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240b2850 con 0x7f0524160340 2023-10-19T10:52:56.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f05240b1f30 0x7f05240b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.690 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f05240b1f30 0x7f05240b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39270/0 (socket says 172.21.15.50:39270) 2023-10-19T10:52:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 -- 172.21.15.50:0/3829893130 learned_addr learned my addr 172.21.15.50:0/3829893130 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f0522d9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f052406f270 0x7f052406f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42730/0 (socket says 172.21.15.50:42730) 2023-10-19T10:52:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f051bfff640 1 --2- 172.21.15.50:0/3829893130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0524160340 0x7f05241607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 -- 172.21.15.50:0/3829893130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0524160340 msgr2=0x7f05241607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 --2- 172.21.15.50:0/3829893130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0524160340 0x7f05241607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 -- 172.21.15.50:0/3829893130 >> v1:172.21.15.50:6789/0 conn(0x7f052406f270 legacy=0x7f052406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f0521d9b640 1 -- 172.21.15.50:0/3829893130 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805217991 0 0) 0x7f052415ee00 con 0x7f052406f270 2023-10-19T10:52:56.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.684+0000 7f052259c640 1 -- 172.21.15.50:0/3829893130 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052415f030 con 0x7f05240b1f30 2023-10-19T10:52:56.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f052259c640 1 --2- 172.21.15.50:0/3829893130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05240b1f30 0x7f05240b2310 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0514009870 tx=0x7f051402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c09815df819134 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0521d9b640 1 -- 172.21.15.50:0/3829893130 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514031640 con 0x7f05240b1f30 2023-10-19T10:52:56.693 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0521d9b640 1 -- 172.21.15.50:0/3829893130 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140317a0 con 0x7f05240b1f30 2023-10-19T10:52:56.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0521d9b640 1 -- 172.21.15.50:0/3829893130 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514031b30 con 0x7f05240b1f30 2023-10-19T10:52:56.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 -- 172.21.15.50:0/3829893130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05240b1f30 msgr2=0x7f05240b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 --2- 172.21.15.50:0/3829893130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05240b1f30 0x7f05240b2310 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0514009870 tx=0x7f051402b8f0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.694 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 -- 172.21.15.50:0/3829893130 shutdown_connections 2023-10-19T10:52:56.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 --2- 172.21.15.50:0/3829893130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0524160340 0x7f05241607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 --2- 172.21.15.50:0/3829893130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05240b1f30 0x7f05240b2310 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 -- 172.21.15.50:0/3829893130 >> 172.21.15.50:0/3829893130 conn(0x7f05240ccf30 msgr2=0x7f05240cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 -- 172.21.15.50:0/3829893130 shutdown_connections 2023-10-19T10:52:56.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.685+0000 7f0528ee9640 1 -- 172.21.15.50:0/3829893130 wait complete. 2023-10-19T10:52:56.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 Processor -- start 2023-10-19T10:52:56.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 -- start start 2023-10-19T10:52:56.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 0x7f05241459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05240b1f30 0x7f05241e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0524160340 0x7f05240b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524145ee0 con 0x7f052406f270 2023-10-19T10:52:56.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05241e6350 con 0x7f0524160340 2023-10-19T10:52:56.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.687+0000 7f0528ee9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240c82e0 con 0x7f05240b1f30 2023-10-19T10:52:56.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 0x7f05241459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 0x7f05241459a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60184/0 (socket says 172.21.15.50:60184) 2023-10-19T10:52:56.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 -- 172.21.15.50:0/4198295384 learned_addr learned my addr 172.21.15.50:0/4198295384 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:56.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f051bfff640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05240b1f30 0x7f05241e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f0522d9d640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0524160340 0x7f05240b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 -- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05240b1f30 msgr2=0x7f05241e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05240b1f30 0x7f05241e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 -- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0524160340 msgr2=0x7f05240b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0524160340 0x7f05240b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 -- 172.21.15.50:0/4198295384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140094d0 con 0x7f052406f270 2023-10-19T10:52:56.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f052259c640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 0x7f05241459a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0514034040 tx=0x7f05140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514004230 con 0x7f052406f270 2023-10-19T10:52:56.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0514004390 con 0x7f052406f270 2023-10-19T10:52:56.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05240c76d0 con 0x7f052406f270 2023-10-19T10:52:56.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051403aa90 con 0x7f052406f270 2023-10-19T10:52:56.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.688+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05240c7c90 con 0x7f052406f270 2023-10-19T10:52:56.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.689+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0524073990 con 0x7f052406f270 2023-10-19T10:52:56.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.690+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0514042070 con 0x7f052406f270 2023-10-19T10:52:56.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.690+0000 7f051affd640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f050406a050 0x7f050406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:56.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.690+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f05140aa180 con 0x7f052406f270 2023-10-19T10:52:56.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.690+0000 7f051bfff640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f050406a050 0x7f050406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:56.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.691+0000 7f051bfff640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f050406a050 0x7f050406c510 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f05240c3b90 tx=0x7f050c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:56.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.695+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0514075a00 con 0x7f052406f270 2023-10-19T10:52:56.872 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.869+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f052415d090 con 0x7f050406a050 2023-10-19T10:52:56.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.870+0000 7f051affd640 1 -- 172.21.15.50:0/4198295384 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f050c00c040 con 0x7f050406a050 2023-10-19T10:52:56.873 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 4 to deep-scrub 2023-10-19T10:52:56.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f050406a050 msgr2=0x7f050406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f050406a050 0x7f050406c510 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f05240c3b90 tx=0x7f050c008040 comp rx=0 tx=0).stop 2023-10-19T10:52:56.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 msgr2=0x7f05241459a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:56.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 0x7f05241459a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0514034040 tx=0x7f05140046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 shutdown_connections 2023-10-19T10:52:56.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f050406a050 0x7f050406c510 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.876 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0524160340 0x7f05240b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f05240b1f30 0x7f05241e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 --2- 172.21.15.50:0/4198295384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052406f270 0x7f05241459a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:56.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.871+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 >> 172.21.15.50:0/4198295384 conn(0x7f05240ccf30 msgr2=0x7f052406f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:56.877 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.872+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 shutdown_connections 2023-10-19T10:52:56.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.872+0000 7f0528ee9640 1 -- 172.21.15.50:0/4198295384 wait complete. 2023-10-19T10:52:56.885 INFO:tasks.ceph:Scrubbing osd.5 2023-10-19T10:52:56.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:56.998 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.995+0000 7f5eb5c10640 1 Processor -- start 2023-10-19T10:52:56.999 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.995+0000 7f5eb5c10640 1 -- start start 2023-10-19T10:52:57.002 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eb5c10640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5eb015c620 0x7f5eb00b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eb5c10640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 0x7f5eb00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eb5c10640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00cda30 con 0x7f5eb00b55c0 2023-10-19T10:52:57.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eb5c10640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00ccc10 con 0x7f5eb015c620 2023-10-19T10:52:57.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eb5c10640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0161110 con 0x7f5eb00b82f0 2023-10-19T10:52:57.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eaffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5eb00b55c0 0x7f5eb00b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57588/0 (socket says 172.21.15.50:57588) 2023-10-19T10:52:57.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.996+0000 7f5eaffff640 1 -- 172.21.15.50:0/613661309 learned_addr learned my addr 172.21.15.50:0/613661309 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:57.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/613661309 >> v2:172.21.15.183:3300/0 conn(0x7f5eb015c620 0x7f5eb00b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaeffd640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 0x7f5eb00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371104363 0 0) 0x7f5eb00cda30 con 0x7f5eb00b55c0 2023-10-19T10:52:57.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e98003520 con 0x7f5eb00b55c0 2023-10-19T10:52:57.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481143146 0 0) 0x7f5e98003520 con 0x7f5eb00b55c0 2023-10-19T10:52:57.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 msgr2=0x7f5eb00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 0x7f5eb00cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 >> v2:172.21.15.183:3300/0 conn(0x7f5eb015c620 msgr2=0x7f5eb00b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 --2- 172.21.15.50:0/613661309 >> v2:172.21.15.183:3300/0 conn(0x7f5eb015c620 0x7f5eb00b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00cce20 con 0x7f5eb00b55c0 2023-10-19T10:52:57.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f5ea80019f0 con 0x7f5eb00b55c0 2023-10-19T10:52:57.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 >> v1:172.21.15.50:6789/0 conn(0x7f5eb00b55c0 legacy=0x7f5eb00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98003970 0x7f5e98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 0x7f5e980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e98004d00 0x7f5e98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00ccc10 con 0x7f5e98003970 2023-10-19T10:52:57.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0161110 con 0x7f5e98004d00 2023-10-19T10:52:57.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9800b980 con 0x7f5eb00b82f0 2023-10-19T10:52:57.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98003970 0x7f5e98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaeffd640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 0x7f5e980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.010 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaffff640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e98004d00 0x7f5e98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 -- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 msgr2=0x7f5e980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5eb00b82f0 0x7f5e980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 -- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e98004d00 msgr2=0x7f5e98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e98004d00 0x7f5e98005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaf7fe640 1 -- 172.21.15.50:0/613661309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98003520 con 0x7f5e98003970 2023-10-19T10:52:57.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.997+0000 7f5eaffff640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e98004d00 0x7f5e98005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:57.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.998+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98003970 0x7f5e98003d90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009ac0 tx=0x7f5e980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82075f6cdbf86bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.998+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea002c710 con 0x7f5e98003970 2023-10-19T10:52:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.998+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea002c870 con 0x7f5e98003970 2023-10-19T10:52:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.998+0000 7f5eae7fc640 1 -- 172.21.15.50:0/613661309 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea002cc00 con 0x7f5e98003970 2023-10-19T10:52:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eb5c10640 1 -- 172.21.15.50:0/613661309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98003970 msgr2=0x7f5e98003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eb5c10640 1 --2- 172.21.15.50:0/613661309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98003970 0x7f5e98003d90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009ac0 tx=0x7f5e980057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eaffff640 1 -- 172.21.15.50:0/613661309 reap_dead start 2023-10-19T10:52:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eb5c10640 1 -- 172.21.15.50:0/613661309 shutdown_connections 2023-10-19T10:52:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eb5c10640 1 -- 172.21.15.50:0/613661309 >> 172.21.15.50:0/613661309 conn(0x7f5eb0071110 msgr2=0x7f5eb00714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eb5c10640 1 -- 172.21.15.50:0/613661309 shutdown_connections 2023-10-19T10:52:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:56.999+0000 7f5eb5c10640 1 -- 172.21.15.50:0/613661309 wait complete. 2023-10-19T10:52:57.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.000+0000 7f5eb5c10640 1 Processor -- start 2023-10-19T10:52:57.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.000+0000 7f5eb5c10640 1 -- start start 2023-10-19T10:52:57.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eb5c10640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e98003970 0x7f5eb0154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eb5c10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5eb0150b60 0x7f5eb00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eb5c10640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5eb00b6f80 0x7f5eb00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eb5c10640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00b7940 con 0x7f5eb00b6f80 2023-10-19T10:52:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eb5c10640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb015e4a0 con 0x7f5eb0150b60 2023-10-19T10:52:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eb5c10640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cc3e0 con 0x7f5e98003970 2023-10-19T10:52:57.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5eb00b6f80 0x7f5eb00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5eb00b6f80 0x7f5eb00b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60200/0 (socket says 172.21.15.50:60200) 2023-10-19T10:52:57.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 -- 172.21.15.50:0/3088665321 learned_addr learned my addr 172.21.15.50:0/3088665321 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:57.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e98003970 0x7f5eb0154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.023 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaeffd640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5eb0150b60 0x7f5eb00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 -- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e98003970 msgr2=0x7f5eb0154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e98003970 0x7f5eb0154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.024 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 -- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5eb0150b60 msgr2=0x7f5eb00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5eb0150b60 0x7f5eb00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaffff640 1 -- 172.21.15.50:0/3088665321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0150fc0 con 0x7f5eb00b6f80 2023-10-19T10:52:57.025 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.001+0000 7f5eaeffd640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5eb0150b60 0x7f5eb00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:57.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.002+0000 7f5eaffff640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5eb00b6f80 0x7f5eb00b7400 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5eb006b290 tx=0x7f5ea800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.002+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8007a80 con 0x7f5eb00b6f80 2023-10-19T10:52:57.026 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.002+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800eda0 con 0x7f5eb00b6f80 2023-10-19T10:52:57.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.002+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb00cc940 con 0x7f5eb00b6f80 2023-10-19T10:52:57.027 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.002+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8013660 con 0x7f5eb00b6f80 2023-10-19T10:52:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.002+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb015de50 con 0x7f5eb00b6f80 2023-10-19T10:52:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.003+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5eb00cd3e0 con 0x7f5eb00b6f80 2023-10-19T10:52:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.003+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5ea80137c0 con 0x7f5eb00b6f80 2023-10-19T10:52:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5e97fff640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5e9c06a050 0x7f5e9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f5ea8088810 con 0x7f5eb00b6f80 2023-10-19T10:52:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5e97fff640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f5e9c0709c0 0x7f5e9c072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 --> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e9c0734d0 con 0x7f5e9c0709c0 2023-10-19T10:52:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea808d2c0 con 0x7f5eb00b6f80 2023-10-19T10:52:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5e9c06a050 0x7f5e9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5eaeffd640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f5e9c0709c0 0x7f5e9c072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.030 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5eaf7fe640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5e9c06a050 0x7f5e9c06c510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0036000 tx=0x7f5ea0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.004+0000 7f5eaeffd640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f5e9c0709c0 0x7f5e9c072e00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.005+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== osd.5 v2:172.21.15.183:6808/3486412186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5e9c0734d0 con 0x7f5e9c0709c0 2023-10-19T10:52:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.018+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 --> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5eb00cd100 con 0x7f5e9c0709c0 2023-10-19T10:52:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.018+0000 7f5e97fff640 1 -- 172.21.15.50:0/3088665321 <== osd.5 v2:172.21.15.183:6808/3486412186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5ea4009c00 con 0x7f5e9c0709c0 2023-10-19T10:52:57.033 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:57.033 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:57.033 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f5e9c0709c0 msgr2=0x7f5e9c072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.035 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6808/3486412186,v1:172.21.15.183:6809/3486412186] conn(0x7f5e9c0709c0 0x7f5e9c072e00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5e9c06a050 msgr2=0x7f5e9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f5e9c06a050 0x7f5e9c06c510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0036000 tx=0x7f5ea0030000 comp rx=0 tx=0).stop 2023-10-19T10:52:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5eb00b6f80 msgr2=0x7f5eb00b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 --2- 172.21.15.50:0/3088665321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5eb00b6f80 0x7f5eb00b7400 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5eb006b290 tx=0x7f5ea800e530 comp rx=0 tx=0).stop 2023-10-19T10:52:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eaffff640 1 -- 172.21.15.50:0/3088665321 reap_dead start 2023-10-19T10:52:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 shutdown_connections 2023-10-19T10:52:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 >> 172.21.15.50:0/3088665321 conn(0x7f5eb0071110 msgr2=0x7f5eb00d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 shutdown_connections 2023-10-19T10:52:57.039 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.019+0000 7f5eb5c10640 1 -- 172.21.15.50:0/3088665321 wait complete. 2023-10-19T10:52:57.039 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-19T10:52:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.144+0000 7efdffdeb640 1 Processor -- start 2023-10-19T10:52:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.144+0000 7efdffdeb640 1 -- start start 2023-10-19T10:52:57.150 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdffdeb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efdf8156510 0x7efdf80ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdffdeb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 0x7efdf80cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdffdeb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf80c95d0 con 0x7efdf80cadc0 2023-10-19T10:52:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdffdeb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf806f770 con 0x7efdf8156510 2023-10-19T10:52:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdffdeb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80cb270 con 0x7efdf8159920 2023-10-19T10:52:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfd386640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7efdf80cadc0 0x7efdf8159210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57594/0 (socket says 172.21.15.50:57594) 2023-10-19T10:52:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfd386640 1 -- 172.21.15.50:0/252686644 learned_addr learned my addr 172.21.15.50:0/252686644 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfe388640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 0x7efdf80cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279684033 0 0) 0x7efdf80c95d0 con 0x7efdf80cadc0 2023-10-19T10:52:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde4004630 con 0x7efdf80cadc0 2023-10-19T10:52:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfdb87640 1 --2- 172.21.15.50:0/252686644 >> v2:172.21.15.183:3300/0 conn(0x7efdf8156510 0x7efdf80ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2892367157 0 0) 0x7efde4004630 con 0x7efdf80cadc0 2023-10-19T10:52:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 msgr2=0x7efdf80cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 0x7efdf80cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 >> v2:172.21.15.183:3300/0 conn(0x7efdf8156510 msgr2=0x7efdf80ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 --2- 172.21.15.50:0/252686644 >> v2:172.21.15.183:3300/0 conn(0x7efdf8156510 0x7efdf80ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf80c8ca0 con 0x7efdf80cadc0 2023-10-19T10:52:57.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7efde8001670 con 0x7efdf80cadc0 2023-10-19T10:52:57.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 >> v1:172.21.15.50:6789/0 conn(0x7efdf80cadc0 legacy=0x7efdf8159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.155 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 0x7efde4004a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde4005000 0x7efde40058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.145+0000 7efdfcb85640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4005e10 0x7efde4006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80cb270 con 0x7efde4005e10 2023-10-19T10:52:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde400ca90 con 0x7efde4005000 2023-10-19T10:52:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde400cbd0 con 0x7efdf8159920 2023-10-19T10:52:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfe388640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4005e10 0x7efde4006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfdb87640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 0x7efde4004a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfd386640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde4005000 0x7efde40058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfe388640 1 -- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 msgr2=0x7efde4004a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfe388640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf8159920 0x7efde4004a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfe388640 1 -- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde4005000 msgr2=0x7efde40058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfe388640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde4005000 0x7efde40058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.159 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.146+0000 7efdfe388640 1 -- 172.21.15.50:0/252686644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4004630 con 0x7efde4005e10 2023-10-19T10:52:57.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdfe388640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4005e10 0x7efde4006290 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009fc0 tx=0x7efde4006900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bc0b234c0fc254 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf002cc00 con 0x7efde4005e10 2023-10-19T10:52:57.160 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000f040 con 0x7efde4005e10 2023-10-19T10:52:57.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdfcb85640 1 -- 172.21.15.50:0/252686644 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0034440 con 0x7efde4005e10 2023-10-19T10:52:57.161 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdffdeb640 1 -- 172.21.15.50:0/252686644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4005e10 msgr2=0x7efde4006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdffdeb640 1 --2- 172.21.15.50:0/252686644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efde4005e10 0x7efde4006290 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009fc0 tx=0x7efde4006900 comp rx=0 tx=0).stop 2023-10-19T10:52:57.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdfe388640 1 -- 172.21.15.50:0/252686644 reap_dead start 2023-10-19T10:52:57.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdffdeb640 1 -- 172.21.15.50:0/252686644 shutdown_connections 2023-10-19T10:52:57.162 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.147+0000 7efdffdeb640 1 -- 172.21.15.50:0/252686644 >> 172.21.15.50:0/252686644 conn(0x7efdf8069e90 msgr2=0x7efdf80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.148+0000 7efdffdeb640 1 -- 172.21.15.50:0/252686644 shutdown_connections 2023-10-19T10:52:57.163 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.148+0000 7efdffdeb640 1 -- 172.21.15.50:0/252686644 wait complete. 2023-10-19T10:52:57.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 Processor -- start 2023-10-19T10:52:57.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 -- start start 2023-10-19T10:52:57.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 0x7efdf80af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.166 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf80c4870 0x7efdf80ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf80c4320 0x7efdf80abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80702b0 con 0x7efdf80aed90 2023-10-19T10:52:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80c80b0 con 0x7efdf80c4870 2023-10-19T10:52:57.167 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdffdeb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf806ff10 con 0x7efdf80c4320 2023-10-19T10:52:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdfdb87640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 0x7efdf80af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.149+0000 7efdfdb87640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 0x7efdf80af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60208/0 (socket says 172.21.15.50:60208) 2023-10-19T10:52:57.168 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 -- 172.21.15.50:0/1069336138 learned_addr learned my addr 172.21.15.50:0/1069336138 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:57.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfd386640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf80c4870 0x7efdf80ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfe388640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf80c4320 0x7efdf80abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 -- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf80c4320 msgr2=0x7efdf80abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.169 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf80c4320 0x7efdf80abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 -- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf80c4870 msgr2=0x7efdf80ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf80c4870 0x7efdf80ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 -- 172.21.15.50:0/1069336138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf80c7af0 con 0x7efdf80aed90 2023-10-19T10:52:57.170 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdfdb87640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 0x7efdf80af170 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efdf4011aa0 tx=0x7efdf4011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4009680 con 0x7efdf80aed90 2023-10-19T10:52:57.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400fac0 con 0x7efdf80aed90 2023-10-19T10:52:57.171 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf80c7dd0 con 0x7efdf80aed90 2023-10-19T10:52:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400fe50 con 0x7efdf80aed90 2023-10-19T10:52:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.150+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf80c4040 con 0x7efdf80aed90 2023-10-19T10:52:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.151+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf8155870 con 0x7efdf80aed90 2023-10-19T10:52:57.172 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.151+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efdf4014440 con 0x7efdf80aed90 2023-10-19T10:52:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.152+0000 7efdee7fc640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efddc06a070 0x7efddc06c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.152+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7efdf4088cb0 con 0x7efdf80aed90 2023-10-19T10:52:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.152+0000 7efdfd386640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efddc06a070 0x7efddc06c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.173 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.153+0000 7efdfd386640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efddc06a070 0x7efddc06c530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efdf80ad500 tx=0x7efde80078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.174 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.156+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7efdf4057fc0 con 0x7efdf80aed90 2023-10-19T10:52:57.333 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.330+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7efdf80c9260 con 0x7efddc06a070 2023-10-19T10:52:57.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.331+0000 7efdee7fc640 1 -- 172.21.15.50:0/1069336138 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efde8007690 con 0x7efddc06a070 2023-10-19T10:52:57.334 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 5 to deep-scrub 2023-10-19T10:52:57.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.332+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efddc06a070 msgr2=0x7efddc06c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.332+0000 7efdffdeb640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efddc06a070 0x7efddc06c530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efdf80ad500 tx=0x7efde80078f0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.332+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 msgr2=0x7efdf80af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.332+0000 7efdffdeb640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 0x7efdf80af170 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efdf4011aa0 tx=0x7efdf4011ea0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 shutdown_connections 2023-10-19T10:52:57.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7efddc06a070 0x7efddc06c530 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efdf80c4320 0x7efdf80abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf80c4870 0x7efdf80ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 --2- 172.21.15.50:0/1069336138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf80aed90 0x7efdf80af170 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 >> 172.21.15.50:0/1069336138 conn(0x7efdf8069e90 msgr2=0x7efdf815c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 shutdown_connections 2023-10-19T10:52:57.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.333+0000 7efdffdeb640 1 -- 172.21.15.50:0/1069336138 wait complete. 2023-10-19T10:52:57.346 INFO:tasks.ceph:Scrubbing osd.6 2023-10-19T10:52:57.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.453+0000 7f4623078640 1 Processor -- start 2023-10-19T10:52:57.457 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4623078640 1 -- start start 2023-10-19T10:52:57.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4623078640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f461c0cfb00 0x7f461c167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4623078640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0cda30 con 0x7f461c15c620 2023-10-19T10:52:57.460 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4623078640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c161110 con 0x7f461c0b55c0 2023-10-19T10:52:57.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4623078640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0ccc10 con 0x7f461c0cfb00 2023-10-19T10:52:57.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4620e14640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f461c15c620 0x7f461c0b7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57608/0 (socket says 172.21.15.50:57608) 2023-10-19T10:52:57.461 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.454+0000 7f4620e14640 1 -- 172.21.15.50:0/2740032539 learned_addr learned my addr 172.21.15.50:0/2740032539 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:57.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f4621615640 1 --2- 172.21.15.50:0/2740032539 >> v2:172.21.15.183:3300/0 conn(0x7f461c0cfb00 0x7f461c167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913273390 0 0) 0x7f461c0cda30 con 0x7f461c15c620 2023-10-19T10:52:57.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f460c003520 con 0x7f461c15c620 2023-10-19T10:52:57.462 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 6014749 0 0) 0x7f460c003520 con 0x7f461c15c620 2023-10-19T10:52:57.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 >> v2:172.21.15.183:3300/0 conn(0x7f461c0cfb00 msgr2=0x7f461c167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 --2- 172.21.15.50:0/2740032539 >> v2:172.21.15.183:3300/0 conn(0x7f461c0cfb00 0x7f461c167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 >> v1:172.21.15.183:6789/0 conn(0x7f461c0b55c0 legacy=0x7f461c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.463 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0cce20 con 0x7f461c15c620 2023-10-19T10:52:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f4610001670 con 0x7f461c15c620 2023-10-19T10:52:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 >> v1:172.21.15.50:6789/0 conn(0x7f461c15c620 legacy=0x7f461c0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f460c003970 0x7f460c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f460c0047c0 0x7f460c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c005180 0x7f460c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0ccc10 con 0x7f460c003970 2023-10-19T10:52:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c00c020 con 0x7f460c005180 2023-10-19T10:52:57.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c00c1e0 con 0x7f460c0047c0 2023-10-19T10:52:57.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f4620e14640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f460c003970 0x7f460c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f461bfff640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f460c0047c0 0x7f460c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.466 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.455+0000 7f4621615640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c005180 0x7f460c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4620e14640 1 -- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f460c0047c0 msgr2=0x7f460c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4620e14640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f460c0047c0 0x7f460c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4620e14640 1 -- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c005180 msgr2=0x7f460c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.467 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4620e14640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c005180 0x7f460c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4620e14640 1 -- 172.21.15.50:0/2740032539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c003520 con 0x7f460c003970 2023-10-19T10:52:57.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4621615640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c005180 0x7f460c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:57.468 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4621615640 1 -- 172.21.15.50:0/2740032539 reap_dead start 2023-10-19T10:52:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f4620e14640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f460c003970 0x7f460c003d90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f461000c190 tx=0x7f460c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d46fd94ea80e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610005ad0 con 0x7f460c003970 2023-10-19T10:52:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610005c30 con 0x7f460c003970 2023-10-19T10:52:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.456+0000 7f461b7fe640 1 -- 172.21.15.50:0/2740032539 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461002cab0 con 0x7f460c003970 2023-10-19T10:52:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 -- 172.21.15.50:0/2740032539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f460c003970 msgr2=0x7f460c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f460c003970 0x7f460c003d90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f461000c190 tx=0x7f460c0057f0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 -- 172.21.15.50:0/2740032539 shutdown_connections 2023-10-19T10:52:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 --2- 172.21.15.50:0/2740032539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f460c003970 0x7f460c003d90 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 -- 172.21.15.50:0/2740032539 >> 172.21.15.50:0/2740032539 conn(0x7f461c071110 msgr2=0x7f461c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 -- 172.21.15.50:0/2740032539 shutdown_connections 2023-10-19T10:52:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.457+0000 7f4623078640 1 -- 172.21.15.50:0/2740032539 wait complete. 2023-10-19T10:52:57.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.458+0000 7f4623078640 1 Processor -- start 2023-10-19T10:52:57.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.458+0000 7f4623078640 1 -- start start 2023-10-19T10:52:57.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4623078640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c003970 0x7f461c148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4623078640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f461c0ab7b0 0x7f461c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.475 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4623078640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461c0ca870 0x7f461c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4623078640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0ac170 con 0x7f461c0ca870 2023-10-19T10:52:57.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4623078640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c148db0 con 0x7f460c003970 2023-10-19T10:52:57.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4623078640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c9ab0 con 0x7f461c0ab7b0 2023-10-19T10:52:57.476 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4621615640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461c0ca870 0x7f461c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4621615640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461c0ca870 0x7f461c0cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60234/0 (socket says 172.21.15.50:60234) 2023-10-19T10:52:57.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.459+0000 7f4621615640 1 -- 172.21.15.50:0/1499386013 learned_addr learned my addr 172.21.15.50:0/1499386013 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:57.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4620e14640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c003970 0x7f461c148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.477 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f461bfff640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f461c0ab7b0 0x7f461c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4621615640 1 -- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f461c0ab7b0 msgr2=0x7f461c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4621615640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f461c0ab7b0 0x7f461c0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.478 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4621615640 1 -- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c003970 msgr2=0x7f461c148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4621615640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c003970 0x7f461c148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4621615640 1 -- 172.21.15.50:0/1499386013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0c9d90 con 0x7f461c0ca870 2023-10-19T10:52:57.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4620e14640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c003970 0x7f461c148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:57.479 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4621615640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461c0ca870 0x7f461c0cacf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f461c0d2570 tx=0x7f46140115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e070 con 0x7f461c0ca870 2023-10-19T10:52:57.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614011c00 con 0x7f461c0ca870 2023-10-19T10:52:57.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0ca070 con 0x7f461c0ca870 2023-10-19T10:52:57.480 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614011db0 con 0x7f461c0ca870 2023-10-19T10:52:57.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.460+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c0cb230 con 0x7f461c0ca870 2023-10-19T10:52:57.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.461+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f461c0cd3e0 con 0x7f461c0ca870 2023-10-19T10:52:57.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.461+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4614003990 con 0x7f461c0ca870 2023-10-19T10:52:57.481 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f46197fa640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f460006a080 0x7f460006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f46140889a0 con 0x7f461c0ca870 2023-10-19T10:52:57.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f4620e14640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f460006a080 0x7f460006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.482 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f46197fa640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7f46000709f0 0x7f4600072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 --> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4600073500 con 0x7f46000709f0 2023-10-19T10:52:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f461408d2c0 con 0x7f461c0ca870 2023-10-19T10:52:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f461bfff640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7f46000709f0 0x7f4600072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.483 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.462+0000 7f4620e14640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f460006a080 0x7f460006c540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4610034000 tx=0x7f4610009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.463+0000 7f461bfff640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7f46000709f0 0x7f4600072e30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.484 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.463+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== osd.6 v2:172.21.15.183:6816/265753870 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4600073500 con 0x7f46000709f0 2023-10-19T10:52:57.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.476+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 --> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f461c0cd100 con 0x7f46000709f0 2023-10-19T10:52:57.485 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f46197fa640 1 -- 172.21.15.50:0/1499386013 <== osd.6 v2:172.21.15.183:6816/265753870 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4608003750 con 0x7f46000709f0 2023-10-19T10:52:57.485 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:57.486 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:57.487 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7f46000709f0 msgr2=0x7f4600072e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6816/265753870,v1:172.21.15.183:6817/265753870] conn(0x7f46000709f0 0x7f4600072e30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f460006a080 msgr2=0x7f460006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f460006a080 0x7f460006c540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4610034000 tx=0x7f4610009030 comp rx=0 tx=0).stop 2023-10-19T10:52:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461c0ca870 msgr2=0x7f461c0cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 --2- 172.21.15.50:0/1499386013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461c0ca870 0x7f461c0cacf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f461c0d2570 tx=0x7f46140115a0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.490 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4621615640 1 -- 172.21.15.50:0/1499386013 reap_dead start 2023-10-19T10:52:57.491 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 shutdown_connections 2023-10-19T10:52:57.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.477+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 >> 172.21.15.50:0/1499386013 conn(0x7f461c071110 msgr2=0x7f461c15f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.478+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 shutdown_connections 2023-10-19T10:52:57.492 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.478+0000 7f4623078640 1 -- 172.21.15.50:0/1499386013 wait complete. 2023-10-19T10:52:57.494 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-19T10:52:57.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.598+0000 7f9eef02f640 1 Processor -- start 2023-10-19T10:52:57.601 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.598+0000 7f9eef02f640 1 -- start start 2023-10-19T10:52:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eef02f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee815c570 0x7f9ee80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eef02f640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9ee815c950 0x7f9ee80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eef02f640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee80cde60 con 0x7f9ee806b760 2023-10-19T10:52:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eef02f640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee806aa30 con 0x7f9ee815c950 2023-10-19T10:52:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eef02f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee806b2d0 con 0x7f9ee815c570 2023-10-19T10:52:57.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eecdcb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee815c570 0x7f9ee80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9ee7fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9ee815c950 0x7f9ee80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.605 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eed5cc640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f9ee806b760 0x7f9ee80d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.50:59612/0 (socket says 172.21.15.50:59612) 2023-10-19T10:52:57.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eed5cc640 1 -- 172.21.15.50:0/3583525806 learned_addr learned my addr 172.21.15.50:0/3583525806 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:57.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9ee77fe640 1 -- 172.21.15.50:0/3583525806 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855180706 0 0) 0x7f9ee80cde60 con 0x7f9ee806b760 2023-10-19T10:52:57.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9ee77fe640 1 -- 172.21.15.50:0/3583525806 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed8003520 con 0x7f9ee806b760 2023-10-19T10:52:57.606 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eecdcb640 1 -- 172.21.15.50:0/3583525806 >> v2:172.21.15.183:3300/0 conn(0x7f9ee815c950 msgr2=0x7f9ee80cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eecdcb640 1 --2- 172.21.15.50:0/3583525806 >> v2:172.21.15.183:3300/0 conn(0x7f9ee815c950 0x7f9ee80cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eecdcb640 1 -- 172.21.15.50:0/3583525806 >> v1:172.21.15.183:6789/0 conn(0x7f9ee806b760 legacy=0x7f9ee80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9eecdcb640 1 -- 172.21.15.50:0/3583525806 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee80ce090 con 0x7f9ee815c570 2023-10-19T10:52:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.599+0000 7f9ee7fff640 1 --2- 172.21.15.50:0/3583525806 >> v2:172.21.15.183:3300/0 conn(0x7f9ee815c950 0x7f9ee80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:52:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.600+0000 7f9eecdcb640 1 --2- 172.21.15.50:0/3583525806 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee815c570 0x7f9ee80d0e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009790 tx=0x7f9edc02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed8b3a4ee9523d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.600+0000 7f9ee77fe640 1 -- 172.21.15.50:0/3583525806 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc031770 con 0x7f9ee815c570 2023-10-19T10:52:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.600+0000 7f9ee77fe640 1 -- 172.21.15.50:0/3583525806 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc0318d0 con 0x7f9ee815c570 2023-10-19T10:52:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.600+0000 7f9ee77fe640 1 -- 172.21.15.50:0/3583525806 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc031c40 con 0x7f9ee815c570 2023-10-19T10:52:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 -- 172.21.15.50:0/3583525806 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee815c570 msgr2=0x7f9ee80d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 --2- 172.21.15.50:0/3583525806 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee815c570 0x7f9ee80d0e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009790 tx=0x7f9edc02b810 comp rx=0 tx=0).stop 2023-10-19T10:52:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 -- 172.21.15.50:0/3583525806 shutdown_connections 2023-10-19T10:52:57.612 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 --2- 172.21.15.50:0/3583525806 >> v2:172.21.15.183:3300/0 conn(0x7f9ee815c950 0x7f9ee80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 --2- 172.21.15.50:0/3583525806 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee815c570 0x7f9ee80d0e30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.613 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 -- 172.21.15.50:0/3583525806 >> 172.21.15.50:0/3583525806 conn(0x7f9ee8089ff0 msgr2=0x7f9ee809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 -- 172.21.15.50:0/3583525806 shutdown_connections 2023-10-19T10:52:57.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.601+0000 7f9eef02f640 1 -- 172.21.15.50:0/3583525806 wait complete. 2023-10-19T10:52:57.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.602+0000 7f9eef02f640 1 Processor -- start 2023-10-19T10:52:57.614 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.602+0000 7f9eef02f640 1 -- start start 2023-10-19T10:52:57.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eef02f640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee806b760 0x7f9ee81599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eef02f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 0x7f9ee80c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.615 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eef02f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee815c950 0x7f9ee80c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eef02f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee815e920 con 0x7f9ee815c570 2023-10-19T10:52:57.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eef02f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80c7f60 con 0x7f9ee815c950 2023-10-19T10:52:57.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eef02f640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee815dde0 con 0x7f9ee806b760 2023-10-19T10:52:57.616 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 0x7f9ee80c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 0x7f9ee80c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60238/0 (socket says 172.21.15.50:60238) 2023-10-19T10:52:57.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9ee7fff640 1 -- 172.21.15.50:0/1938675880 learned_addr learned my addr 172.21.15.50:0/1938675880 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:57.617 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eecdcb640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee806b760 0x7f9ee81599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.603+0000 7f9eed5cc640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee815c950 0x7f9ee80c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee7fff640 1 -- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee806b760 msgr2=0x7f9ee81599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.618 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee7fff640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee806b760 0x7f9ee81599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee7fff640 1 -- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee815c950 msgr2=0x7f9ee80c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee7fff640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee815c950 0x7f9ee80c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee7fff640 1 -- 172.21.15.50:0/1938675880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009400 con 0x7f9ee815c570 2023-10-19T10:52:57.619 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee7fff640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 0x7f9ee80c97e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400ebe0 tx=0x7f9ed400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4009760 con 0x7f9ee815c570 2023-10-19T10:52:57.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed400cc30 con 0x7f9ee815c570 2023-10-19T10:52:57.620 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc0093a0 con 0x7f9ee815c570 2023-10-19T10:52:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400eee0 con 0x7f9ee815c570 2023-10-19T10:52:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.604+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee815e360 con 0x7f9ee815c570 2023-10-19T10:52:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.605+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee8073990 con 0x7f9ee815c570 2023-10-19T10:52:57.621 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.605+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9ed4013470 con 0x7f9ee815c570 2023-10-19T10:52:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.606+0000 7f9ee57fa640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ecc06a050 0x7f9ecc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.606+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f9ed4088640 con 0x7f9ee815c570 2023-10-19T10:52:57.622 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.606+0000 7f9eecdcb640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ecc06a050 0x7f9ecc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.623 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.607+0000 7f9eecdcb640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ecc06a050 0x7f9ecc06c510 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9edc036000 tx=0x7f9edc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.624 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.610+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f9ed4053f80 con 0x7f9ee815c570 2023-10-19T10:52:57.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.784+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ee806af60 con 0x7f9ecc06a050 2023-10-19T10:52:57.787 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.784+0000 7f9ee57fa640 1 -- 172.21.15.50:0/1938675880 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9edc031570 con 0x7f9ecc06a050 2023-10-19T10:52:57.788 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 6 to deep-scrub 2023-10-19T10:52:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ecc06a050 msgr2=0x7f9ecc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ecc06a050 0x7f9ecc06c510 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9edc036000 tx=0x7f9edc008030 comp rx=0 tx=0).stop 2023-10-19T10:52:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 msgr2=0x7f9ee80c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 0x7f9ee80c97e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400ebe0 tx=0x7f9ed400c520 comp rx=0 tx=0).stop 2023-10-19T10:52:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 shutdown_connections 2023-10-19T10:52:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f9ecc06a050 0x7f9ecc06c510 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee815c950 0x7f9ee80c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ee815c570 0x7f9ee80c97e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 --2- 172.21.15.50:0/1938675880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ee806b760 0x7f9ee81599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.786+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 >> 172.21.15.50:0/1938675880 conn(0x7f9ee8089ff0 msgr2=0x7f9ee8160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.787+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 shutdown_connections 2023-10-19T10:52:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.787+0000 7f9eef02f640 1 -- 172.21.15.50:0/1938675880 wait complete. 2023-10-19T10:52:57.800 INFO:tasks.ceph:Scrubbing osd.7 2023-10-19T10:52:57.800 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-19T10:52:57.910 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.907+0000 7f28fccf9640 1 Processor -- start 2023-10-19T10:52:57.911 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.907+0000 7f28fccf9640 1 -- start start 2023-10-19T10:52:57.912 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28fccf9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f815c990 0x7f28f80714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28fccf9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f28f806ec60 0x7f28f8070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28fccf9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f28f80719e0 0x7f28f8070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.913 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28fccf9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80cdb60 con 0x7f28f80719e0 2023-10-19T10:52:57.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28fccf9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80cde40 con 0x7f28f806ec60 2023-10-19T10:52:57.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28fccf9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80ce120 con 0x7f28f815c990 2023-10-19T10:52:57.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28f6d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f28f80719e0 0x7f28f8070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.914 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28f6d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f28f80719e0 0x7f28f8070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60252/0 (socket says 172.21.15.50:60252) 2023-10-19T10:52:57.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.908+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3803624365 learned_addr learned my addr 172.21.15.50:0/3803624365 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:57.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f659c640 1 --2- 172.21.15.50:0/3803624365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f815c990 0x7f28f80714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.915 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f5d9b640 1 --2- 172.21.15.50:0/3803624365 >> v2:172.21.15.183:3300/0 conn(0x7f28f806ec60 0x7f28f8070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3803624365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f815c990 msgr2=0x7f28f80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f6d9d640 1 --2- 172.21.15.50:0/3803624365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f815c990 0x7f28f80714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3803624365 >> v2:172.21.15.183:3300/0 conn(0x7f28f806ec60 msgr2=0x7f28f8070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:57.916 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f6d9d640 1 --2- 172.21.15.50:0/3803624365 >> v2:172.21.15.183:3300/0 conn(0x7f28f806ec60 0x7f28f8070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3803624365 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80ce400 con 0x7f28f80719e0 2023-10-19T10:52:57.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.909+0000 7f28f6d9d640 1 --2- 172.21.15.50:0/3803624365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f80719e0 0x7f28f8070ca0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f28f806c460 tx=0x7f28e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56e9310c73ba155 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28f559a640 1 -- 172.21.15.50:0/3803624365 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e802ca70 con 0x7f28f80719e0 2023-10-19T10:52:57.917 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28f559a640 1 -- 172.21.15.50:0/3803624365 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802cbd0 con 0x7f28f80719e0 2023-10-19T10:52:57.918 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28f559a640 1 -- 172.21.15.50:0/3803624365 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80343f0 con 0x7f28f80719e0 2023-10-19T10:52:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 -- 172.21.15.50:0/3803624365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f80719e0 msgr2=0x7f28f8070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3803624365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f80719e0 0x7f28f8070ca0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f28f806c460 tx=0x7f28e802bed0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 -- 172.21.15.50:0/3803624365 shutdown_connections 2023-10-19T10:52:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3803624365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f80719e0 0x7f28f8070ca0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3803624365 >> v2:172.21.15.183:3300/0 conn(0x7f28f806ec60 0x7f28f8070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3803624365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f815c990 0x7f28f80714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.923 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 -- 172.21.15.50:0/3803624365 >> 172.21.15.50:0/3803624365 conn(0x7f28f80cbb30 msgr2=0x7f28f80cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 -- 172.21.15.50:0/3803624365 shutdown_connections 2023-10-19T10:52:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.910+0000 7f28fccf9640 1 -- 172.21.15.50:0/3803624365 wait complete. 2023-10-19T10:52:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 Processor -- start 2023-10-19T10:52:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 -- start start 2023-10-19T10:52:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f806ec60 0x7f28f814a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f80719e0 0x7f28f80c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f815c990 0x7f28f80c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c6dd0 con 0x7f28f815c990 2023-10-19T10:52:57.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c70b0 con 0x7f28f806ec60 2023-10-19T10:52:57.926 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28fccf9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c7390 con 0x7f28f80719e0 2023-10-19T10:52:57.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28f6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f815c990 0x7f28f80c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28f6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f815c990 0x7f28f80c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60268/0 (socket says 172.21.15.50:60268) 2023-10-19T10:52:57.927 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.912+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3201090315 learned_addr learned my addr 172.21.15.50:0/3201090315 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f5d9b640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f80719e0 0x7f28f80c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f80719e0 msgr2=0x7f28f80c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f6d9d640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f28f80719e0 0x7f28f80c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f659c640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f806ec60 0x7f28f814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f806ec60 msgr2=0x7f28f814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f6d9d640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f806ec60 0x7f28f814a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3201090315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8009b30 con 0x7f28f815c990 2023-10-19T10:52:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28f6d9d640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f815c990 0x7f28f80c8980 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036040 tx=0x7f28e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8009510 con 0x7f28f815c990 2023-10-19T10:52:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e803e450 con 0x7f28f815c990 2023-10-19T10:52:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80c7670 con 0x7f28f815c990 2023-10-19T10:52:57.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e803e800 con 0x7f28f815c990 2023-10-19T10:52:57.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.913+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f814a8a0 con 0x7f28f815c990 2023-10-19T10:52:57.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.914+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28f80b1f60 con 0x7f28f815c990 2023-10-19T10:52:57.931 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f28e803e960 con 0x7f28f815c990 2023-10-19T10:52:57.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28eeffd640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f28d806a050 0x7f28d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f28e80ab070 con 0x7f28f815c990 2023-10-19T10:52:57.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28eeffd640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f28d80709c0 0x7f28d8072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:57.932 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 --> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28d80734d0 con 0x7f28d80709c0 2023-10-19T10:52:57.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e80ad730 con 0x7f28f815c990 2023-10-19T10:52:57.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28f659c640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f28d806a050 0x7f28d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.933 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.915+0000 7f28f5d9b640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f28d80709c0 0x7f28d8072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.916+0000 7f28f659c640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f28d806a050 0x7f28d806c510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009970 tx=0x7f28f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.916+0000 7f28f5d9b640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f28d80709c0 0x7f28d8072e00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.917+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== osd.7 v2:172.21.15.183:6824/3403463202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f28d80734d0 con 0x7f28d80709c0 2023-10-19T10:52:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.929+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 --> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f28f80c3650 con 0x7f28d80709c0 2023-10-19T10:52:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28eeffd640 1 -- 172.21.15.50:0/3201090315 <== osd.7 v2:172.21.15.183:6824/3403463202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f28f80c3650 con 0x7f28d80709c0 2023-10-19T10:52:57.936 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2023-10-19T10:52:57.936 INFO:teuthology.orchestra.run.smithi050.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-19T10:52:57.936 INFO:teuthology.orchestra.run.smithi050.stdout:} 2023-10-19T10:52:57.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f28d80709c0 msgr2=0x7f28d8072e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6824/3403463202,v1:172.21.15.183:6825/3403463202] conn(0x7f28d80709c0 0x7f28d8072e00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:57.938 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f28d806a050 msgr2=0x7f28d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f28d806a050 0x7f28d806c510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009970 tx=0x7f28f0008040 comp rx=0 tx=0).stop 2023-10-19T10:52:57.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f815c990 msgr2=0x7f28f80c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:57.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 --2- 172.21.15.50:0/3201090315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28f815c990 0x7f28f80c8980 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036040 tx=0x7f28e8004a40 comp rx=0 tx=0).stop 2023-10-19T10:52:57.939 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28f6d9d640 1 -- 172.21.15.50:0/3201090315 reap_dead start 2023-10-19T10:52:57.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 shutdown_connections 2023-10-19T10:52:57.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 >> 172.21.15.50:0/3201090315 conn(0x7f28f80cbb30 msgr2=0x7f28f815b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:57.940 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 shutdown_connections 2023-10-19T10:52:57.941 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:57.930+0000 7f28fccf9640 1 -- 172.21.15.50:0/3201090315 wait complete. 2023-10-19T10:52:57.943 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-19T10:52:58.050 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.047+0000 7f64e7dde640 1 Processor -- start 2023-10-19T10:52:58.051 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.048+0000 7f64e7dde640 1 -- start start 2023-10-19T10:52:58.052 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.048+0000 7f64e7dde640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64e015c570 0x7f64e00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.048+0000 7f64e7dde640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.048+0000 7f64e7dde640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00cde60 con 0x7f64e006b760 2023-10-19T10:52:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.048+0000 7f64e7dde640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e006aa30 con 0x7f64e015c570 2023-10-19T10:52:58.053 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.048+0000 7f64e7dde640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e006b2d0 con 0x7f64e015c950 2023-10-19T10:52:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e637b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f64e006b760 0x7f64e00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57624/0 (socket says 172.21.15.50:57624) 2023-10-19T10:52:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e637b640 1 -- 172.21.15.50:0/966160740 learned_addr learned my addr 172.21.15.50:0/966160740 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:58.054 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e5b7a640 1 --2- 172.21.15.50:0/966160740 >> v2:172.21.15.183:3300/0 conn(0x7f64e015c570 0x7f64e00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e5379640 1 --2- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e4b78640 1 -- 172.21.15.50:0/966160740 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163845504 0 0) 0x7f64e00cde60 con 0x7f64e006b760 2023-10-19T10:52:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e4b78640 1 -- 172.21.15.50:0/966160740 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64cc003520 con 0x7f64e006b760 2023-10-19T10:52:58.055 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e5b7a640 1 -- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 msgr2=0x7f64e00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e5b7a640 1 --2- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e5b7a640 1 -- 172.21.15.50:0/966160740 >> v1:172.21.15.50:6789/0 conn(0x7f64e006b760 legacy=0x7f64e00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.056 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.049+0000 7f64e5b7a640 1 -- 172.21.15.50:0/966160740 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00ce090 con 0x7f64e015c570 2023-10-19T10:52:58.059 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.050+0000 7f64e5b7a640 1 --2- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e015c570 0x7f64e00d0e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009790 tx=0x7f64dc02b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a424c0f84164791 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:58.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.050+0000 7f64e4b78640 1 -- 172.21.15.50:0/966160740 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031770 con 0x7f64e015c570 2023-10-19T10:52:58.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.050+0000 7f64e4b78640 1 -- 172.21.15.50:0/966160740 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc0318d0 con 0x7f64e015c570 2023-10-19T10:52:58.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.050+0000 7f64e4b78640 1 -- 172.21.15.50:0/966160740 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031c40 con 0x7f64e015c570 2023-10-19T10:52:58.060 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 -- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e015c570 msgr2=0x7f64e00d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 --2- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e015c570 0x7f64e00d0e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009790 tx=0x7f64dc02b810 comp rx=0 tx=0).stop 2023-10-19T10:52:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 -- 172.21.15.50:0/966160740 shutdown_connections 2023-10-19T10:52:58.061 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 --2- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 --2- 172.21.15.50:0/966160740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e015c570 0x7f64e00d0e30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 -- 172.21.15.50:0/966160740 >> 172.21.15.50:0/966160740 conn(0x7f64e0089ff0 msgr2=0x7f64e009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 -- 172.21.15.50:0/966160740 shutdown_connections 2023-10-19T10:52:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.051+0000 7f64e7dde640 1 -- 172.21.15.50:0/966160740 wait complete. 2023-10-19T10:52:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.052+0000 7f64e7dde640 1 Processor -- start 2023-10-19T10:52:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.052+0000 7f64e7dde640 1 -- start start 2023-10-19T10:52:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.052+0000 7f64e7dde640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e006b760 0x7f64e00b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 0x7f64e0148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b15f0 con 0x7f64e015c570 2023-10-19T10:52:58.064 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01491d0 con 0x7f64e006b760 2023-10-19T10:52:58.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e015fe70 con 0x7f64e015c950 2023-10-19T10:52:58.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 0x7f64e0148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 0x7f64e0148c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60278/0 (socket says 172.21.15.50:60278) 2023-10-19T10:52:58.065 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 -- 172.21.15.50:0/943698435 learned_addr learned my addr 172.21.15.50:0/943698435 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:58.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e637b640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5b7a640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e006b760 0x7f64e00b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.066 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 -- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 msgr2=0x7f64e00b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 -- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e006b760 msgr2=0x7f64e00b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e006b760 0x7f64e00b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.067 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 -- 172.21.15.50:0/943698435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0093a0 con 0x7f64e015c570 2023-10-19T10:52:58.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e5379640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 0x7f64e0148c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e9b0 tx=0x7f64d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:58.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0009670 con 0x7f64e015c570 2023-10-19T10:52:58.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000c9a0 con 0x7f64e015c570 2023-10-19T10:52:58.068 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e015f540 con 0x7f64e015c570 2023-10-19T10:52:58.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000cd50 con 0x7f64e015c570 2023-10-19T10:52:58.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.053+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e015fb00 con 0x7f64e015c570 2023-10-19T10:52:58.069 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.054+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e0073990 con 0x7f64e015c570 2023-10-19T10:52:58.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.055+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f64d0013470 con 0x7f64e015c570 2023-10-19T10:52:58.072 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.055+0000 7f64d67fc640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.055+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f64d0088580 con 0x7f64e015c570 2023-10-19T10:52:58.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.055+0000 7f64e5b7a640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.073 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.056+0000 7f64e5b7a640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f64dc036000 tx=0x7f64dc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:58.074 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.060+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f64d0053eb0 con 0x7f64e015c570 2023-10-19T10:52:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.236+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e006af60 con 0x7f64c406a050 2023-10-19T10:52:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.237+0000 7f64d67fc640 1 -- 172.21.15.50:0/943698435 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f64dc031570 con 0x7f64c406a050 2023-10-19T10:52:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:instructed osd(s) 7 to deep-scrub 2023-10-19T10:52:58.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.238+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 msgr2=0x7f64c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.238+0000 7f64e7dde640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f64dc036000 tx=0x7f64dc008030 comp rx=0 tx=0).stop 2023-10-19T10:52:58.242 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.238+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 msgr2=0x7f64e0148c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.238+0000 7f64e7dde640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 0x7f64e0148c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e9b0 tx=0x7f64d000ee70 comp rx=0 tx=0).stop 2023-10-19T10:52:58.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 shutdown_connections 2023-10-19T10:52:58.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f64c406a050 0x7f64c406c510 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.243 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64e015c950 0x7f64e00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e015c570 0x7f64e0148c90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 --2- 172.21.15.50:0/943698435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e006b760 0x7f64e00b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.244 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 >> 172.21.15.50:0/943698435 conn(0x7f64e0089ff0 msgr2=0x7f64e01604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:58.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 shutdown_connections 2023-10-19T10:52:58.245 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.239+0000 7f64e7dde640 1 -- 172.21.15.50:0/943698435 wait complete. 2023-10-19T10:52:58.252 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:52:58.360 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.357+0000 7fc285c9d640 1 Processor -- start 2023-10-19T10:52:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.357+0000 7fc285c9d640 1 -- start start 2023-10-19T10:52:58.362 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc285c9d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc28015c3a0 0x7fc2800ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc285c9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc28006eeb0 0x7fc2800cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc285c9d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2801605c0 con 0x7fc280069800 2023-10-19T10:52:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc285c9d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2801611d0 con 0x7fc28006eeb0 2023-10-19T10:52:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc285c9d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2801608a0 con 0x7fc28015c3a0 2023-10-19T10:52:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc27effd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc28006eeb0 0x7fc2800cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc27ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc280069800 0x7fc28006dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:57632/0 (socket says 172.21.15.50:57632) 2023-10-19T10:52:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc27effd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc28006eeb0 0x7fc2800cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60286/0 (socket says 172.21.15.50:60286) 2023-10-19T10:52:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.358+0000 7fc27ffff640 1 -- 172.21.15.50:0/4168724999 learned_addr learned my addr 172.21.15.50:0/4168724999 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:52:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/4168724999 >> v2:172.21.15.183:3300/0 conn(0x7fc28015c3a0 0x7fc2800ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27effd640 1 -- 172.21.15.50:0/4168724999 >> v2:172.21.15.183:3300/0 conn(0x7fc28015c3a0 msgr2=0x7fc2800ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:52:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27effd640 1 --2- 172.21.15.50:0/4168724999 >> v2:172.21.15.183:3300/0 conn(0x7fc28015c3a0 0x7fc2800ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27effd640 1 -- 172.21.15.50:0/4168724999 >> v1:172.21.15.50:6789/0 conn(0x7fc280069800 legacy=0x7fc28006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27effd640 1 -- 172.21.15.50:0/4168724999 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280160b80 con 0x7fc28006eeb0 2023-10-19T10:52:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/4168724999 >> v2:172.21.15.183:3300/0 conn(0x7fc28015c3a0 0x7fc2800ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-19T10:52:58.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.359+0000 7fc27effd640 1 --2- 172.21.15.50:0/4168724999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2800cd290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc2800667e0 tx=0x7fc27402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df610ecb3cba2e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:58.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc27e7fc640 1 -- 172.21.15.50:0/4168724999 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27402e070 con 0x7fc28006eeb0 2023-10-19T10:52:58.367 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc27e7fc640 1 -- 172.21.15.50:0/4168724999 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2740315a0 con 0x7fc28006eeb0 2023-10-19T10:52:58.368 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc27e7fc640 1 -- 172.21.15.50:0/4168724999 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274031930 con 0x7fc28006eeb0 2023-10-19T10:52:58.370 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 -- 172.21.15.50:0/4168724999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 msgr2=0x7fc2800cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 --2- 172.21.15.50:0/4168724999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2800cd290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc2800667e0 tx=0x7fc27402b840 comp rx=0 tx=0).stop 2023-10-19T10:52:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 -- 172.21.15.50:0/4168724999 shutdown_connections 2023-10-19T10:52:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 --2- 172.21.15.50:0/4168724999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2800cd290 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 --2- 172.21.15.50:0/4168724999 >> v2:172.21.15.183:3300/0 conn(0x7fc28015c3a0 0x7fc2800ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 -- 172.21.15.50:0/4168724999 >> 172.21.15.50:0/4168724999 conn(0x7fc2800cbc00 msgr2=0x7fc2800cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:58.372 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 -- 172.21.15.50:0/4168724999 shutdown_connections 2023-10-19T10:52:58.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.360+0000 7fc285c9d640 1 -- 172.21.15.50:0/4168724999 wait complete. 2023-10-19T10:52:58.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.361+0000 7fc285c9d640 1 Processor -- start 2023-10-19T10:52:58.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 -- start start 2023-10-19T10:52:58.373 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc280069800 0x7fc2800b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2801535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc28015c3a0 0x7fc2800af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.374 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28015f6b0 con 0x7fc28006eeb0 2023-10-19T10:52:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800b4bf0 con 0x7fc280069800 2023-10-19T10:52:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc285c9d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc280153b10 con 0x7fc28015c3a0 2023-10-19T10:52:58.375 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc27effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2801535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc27effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2801535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60300/0 (socket says 172.21.15.50:60300) 2023-10-19T10:52:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.362+0000 7fc27effd640 1 -- 172.21.15.50:0/1503723950 learned_addr learned my addr 172.21.15.50:0/1503723950 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:52:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27ffff640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc28015c3a0 0x7fc2800af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.376 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc280069800 0x7fc2800b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27effd640 1 -- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc28015c3a0 msgr2=0x7fc2800af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27effd640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc28015c3a0 0x7fc2800af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27effd640 1 -- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc280069800 msgr2=0x7fc2800b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.377 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27effd640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc280069800 0x7fc2800b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27effd640 1 -- 172.21.15.50:0/1503723950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009400 con 0x7fc28006eeb0 2023-10-19T10:52:58.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27ffff640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc28015c3a0 0x7fc2800af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:52:58.378 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc27effd640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2801535d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc274000c00 tx=0x7fc2740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274032840 con 0x7fc28006eeb0 2023-10-19T10:52:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2740329a0 con 0x7fc28006eeb0 2023-10-19T10:52:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28015f0f0 con 0x7fc28006eeb0 2023-10-19T10:52:58.379 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274032d30 con 0x7fc28006eeb0 2023-10-19T10:52:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.363+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2800ae120 con 0x7fc28006eeb0 2023-10-19T10:52:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.364+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2800adf90 con 0x7fc28006eeb0 2023-10-19T10:52:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.365+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc27404d3f0 con 0x7fc28006eeb0 2023-10-19T10:52:58.380 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.365+0000 7fc267fff640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26c06a050 0x7fc26c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:52:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.365+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc2740aa070 con 0x7fc28006eeb0 2023-10-19T10:52:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.365+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26c06a050 0x7fc26c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:52:58.381 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.366+0000 7fc27f7fe640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26c06a050 0x7fc26c06c510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc27000a950 tx=0x7fc270008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:52:58.382 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.369+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fc2740758f0 con 0x7fc28006eeb0 2023-10-19T10:52:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.543+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc280071ac0 con 0x7fc26c06a050 2023-10-19T10:52:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.546+0000 7fc267fff640 1 -- 172.21.15.50:0/1503723950 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72769 (secure 0 0 0) 0x7fc27000c050 con 0x7fc26c06a050 2023-10-19T10:52:58.549 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:52:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:52:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26c06a050 msgr2=0x7fc26c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26c06a050 0x7fc26c06c510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc27000a950 tx=0x7fc270008040 comp rx=0 tx=0).stop 2023-10-19T10:52:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 msgr2=0x7fc2801535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:52:58.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2801535d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc274000c00 tx=0x7fc2740046c0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 shutdown_connections 2023-10-19T10:52:58.553 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fc26c06a050 0x7fc26c06c510 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc28015c3a0 0x7fc2800af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc28006eeb0 0x7fc2801535d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 --2- 172.21.15.50:0/1503723950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc280069800 0x7fc2800b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:52:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 >> 172.21.15.50:0/1503723950 conn(0x7fc2800cbc00 msgr2=0x7fc28006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:52:58.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 shutdown_connections 2023-10-19T10:52:58.555 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:52:58.548+0000 7fc285c9d640 1 -- 172.21.15.50:0/1503723950 wait complete. 2023-10-19T10:52:58.562 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":539,"stamp":"2023-10-19T10:52:56.726412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239620,"kb_used_data":6552,"kb_used_omap":17,"kb_used_meta":232942,"kb_avail":754735100,"statfs":{"total":773094113280,"available":772848742400,"internally_reserved":0,"allocated":6709248,"data_stored":3813600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17557,"internal_metadata":238533483},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.254560"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.481478+0000","last_change":"2023-10-19T10:36:50.170103+0000","last_active":"2023-10-19T10:52:49.481478+0000","last_peered":"2023-10-19T10:52:49.481478+0000","last_clean":"2023-10-19T10:52:49.481478+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:52:49.481478+0000","last_undegraded":"2023-10-19T10:52:49.481478+0000","last_fullsized":"2023-10-19T10:52:49.481478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T16:54:50.979513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.077475+0000","last_change":"2023-10-19T10:36:50.169861+0000","last_active":"2023-10-19T10:52:50.077475+0000","last_peered":"2023-10-19T10:52:50.077475+0000","last_clean":"2023-10-19T10:52:50.077475+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:52:50.077475+0000","last_undegraded":"2023-10-19T10:52:50.077475+0000","last_fullsized":"2023-10-19T10:52:50.077475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.481034+0000","last_change":"2023-10-19T10:36:50.494779+0000","last_active":"2023-10-19T10:52:49.481034+0000","last_peered":"2023-10-19T10:52:49.481034+0000","last_clean":"2023-10-19T10:52:49.481034+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:52:49.481034+0000","last_undegraded":"2023-10-19T10:52:49.481034+0000","last_fullsized":"2023-10-19T10:52:49.481034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:27:22.051906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.075190+0000","last_change":"2023-10-19T10:36:50.494568+0000","last_active":"2023-10-19T10:52:50.075190+0000","last_peered":"2023-10-19T10:52:50.075190+0000","last_clean":"2023-10-19T10:52:50.075190+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:52:50.075190+0000","last_undegraded":"2023-10-19T10:52:50.075190+0000","last_fullsized":"2023-10-19T10:52:50.075190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.075926+0000","last_change":"2023-10-19T10:36:50.167616+0000","last_active":"2023-10-19T10:52:50.075926+0000","last_peered":"2023-10-19T10:52:50.075926+0000","last_clean":"2023-10-19T10:52:50.075926+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:52:50.075926+0000","last_undegraded":"2023-10-19T10:52:50.075926+0000","last_fullsized":"2023-10-19T10:52:50.075926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:41:00.529540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.473502+0000","last_change":"2023-10-19T10:46:26.328460+0000","last_active":"2023-10-19T10:52:49.473502+0000","last_peered":"2023-10-19T10:52:49.473502+0000","last_clean":"2023-10-19T10:52:49.473502+0000","last_became_active":"2023-10-19T10:46:26.328144+0000","last_became_peered":"2023-10-19T10:46:26.328144+0000","last_unstale":"2023-10-19T10:52:49.473502+0000","last_undegraded":"2023-10-19T10:52:49.473502+0000","last_fullsized":"2023-10-19T10:52:49.473502+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T15:58:14.042194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.478645+0000","last_change":"2023-10-19T10:36:50.169721+0000","last_active":"2023-10-19T10:52:49.478645+0000","last_peered":"2023-10-19T10:52:49.478645+0000","last_clean":"2023-10-19T10:52:49.478645+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:52:49.478645+0000","last_undegraded":"2023-10-19T10:52:49.478645+0000","last_fullsized":"2023-10-19T10:52:49.478645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T18:20:02.041855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:50.076673+0000","last_change":"2023-10-19T10:46:26.329138+0000","last_active":"2023-10-19T10:52:50.076673+0000","last_peered":"2023-10-19T10:52:50.076673+0000","last_clean":"2023-10-19T10:52:50.076673+0000","last_became_active":"2023-10-19T10:46:26.328870+0000","last_became_peered":"2023-10-19T10:46:26.328870+0000","last_unstale":"2023-10-19T10:52:50.076673+0000","last_undegraded":"2023-10-19T10:52:50.076673+0000","last_fullsized":"2023-10-19T10:52:50.076673+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:49.152681+0000","last_clean_scrub_stamp":"2023-10-19T10:36:49.152681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T13:25:01.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":251,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:49.478742+0000","last_change":"2023-10-19T10:36:43.651082+0000","last_active":"2023-10-19T10:52:49.478742+0000","last_peered":"2023-10-19T10:52:49.478742+0000","last_clean":"2023-10-19T10:52:49.478742+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:52:49.478742+0000","last_undegraded":"2023-10-19T10:52:49.478742+0000","last_fullsized":"2023-10-19T10:52:49.478742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:36:42.521118+0000","last_clean_scrub_stamp":"2023-10-19T10:36:42.521118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T21:00:51.856854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542342,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":918905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.88400000000000001,"15min":0.91300000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61399999999999999,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75700000000000001,"5min":0.80000000000000004,"15min":0.84599999999999997},"last":0.627}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.53500000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.35099999999999998},"max":{"1min":0.754,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Oct 19 10:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.54100000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73599999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.88,"15min":0.88},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.94499999999999995},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.28699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.313,"15min":0.28999999999999998},"max":{"1min":0.76600000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.28000000000000003},"max":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.84199999999999997},"last":0.433},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.35099999999999998,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42499999999999999,"5min":0.39700000000000002,"15min":0.35499999999999998},"max":{"1min":0.83599999999999997,"5min":0.872,"15min":0.872},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":0.78100000000000003,"5min":0.78300000000000003,"15min":0.83599999999999997},"last":0.46300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575046,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.89900000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.51600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":0.84599999999999997,"15min":0.90700000000000003},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.313},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.83499999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.34499999999999997},"max":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.88},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Oct 19 10:52:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":1.042,"15min":1.042},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.34499999999999997},"max":{"1min":0.77200000000000002,"5min":1.075,"15min":1.075},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.317,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":1.111,"15min":1.111},"last":0.503},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48999999999999999,"15min":0.48499999999999999},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68200000000000005,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48699999999999999,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.499,"15min":0.50800000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.34399999999999997},"max":{"1min":0.753,"5min":0.92600000000000005,"15min":1.02},"last":0.61399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575046,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.312},"max":{"1min":0.747,"5min":0.81599999999999995,"15min":0.997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.32000000000000001},"max":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.93600000000000005},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.33400000000000002},"max":{"1min":0.76100000000000001,"5min":0.83499999999999996,"15min":0.87},"last":0.82599999999999996}]},{"osd":2,"last update":"Thu Oct 19 10:52:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.752,"5min":0.752,"15min":0.79500000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.85599999999999998,"15min":0.91300000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.33800000000000002},"max":{"1min":0.78300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":4,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.27900000000000003},"max":{"1min":0.66800000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.45900000000000002,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.26600000000000001},"max":{"1min":0.65200000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.84199999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80000000000000004},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Oct 19 10:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.89000000000000001,"15min":0.89600000000000002},"last":0.751}]}]},{"osd":4,"up_from":13,"seq":55834575046,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59699999999999998,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78400000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.626},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.33800000000000002},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.89700000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.83299999999999996},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71899999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40100000000000002},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.27800000000000002},"max":{"1min":0.753,"5min":0.75600000000000001,"15min":0.879},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.495}]},{"osd":6,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.94799999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.313},"max":{"1min":0.82999999999999996,"5min":0.91100000000000003,"15min":0.97099999999999997},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Oct 19 10:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.51000000000000001},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.434},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.86199999999999999,"15min":0.96499999999999997},"last":0.52700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575045,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.182},"max":{"1min":0.54800000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.22},"max":{"1min":0.60399999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42099999999999999,"15min":0.40000000000000002},"min":{"1min":0.27500000000000002,"5min":0.247,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.69199999999999995,"15min":0.72199999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.222},"max":{"1min":0.56999999999999995,"5min":0.69399999999999995,"15min":0.74099999999999999},"last":0.372}]},{"osd":2,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46000000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.69999999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70099999999999996,"15min":0.68600000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.36099999999999999},"max":{"1min":0.90300000000000002,"5min":0.92700000000000005,"15min":1.0149999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.66800000000000004},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.376},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.35699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.39000000000000001},"max":{"1min":0.90300000000000002,"5min":0.93700000000000006,"15min":0.997},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.36699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.96099999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.40000000000000002},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":1.0349999999999999},"last":0.5}]},{"osd":7,"last update":"Thu Oct 19 10:52:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69399999999999995,"15min":0.67100000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.98699999999999999},"last":0.624},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51500000000000001}]}]},{"osd":2,"up_from":54,"seq":231928234063,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":676,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5090,"internal_metadata":32369694},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.376,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.313},{"interface":"front","average":{"1min":0.435,"5min":0.432,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":0.70499999999999996},"last":0.26700000000000002}]},{"osd":3,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46700000000000003,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.505},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":1.006,"15min":1.006},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.67900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79700000000000004,"5min":1.05,"15min":1.05},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":0.93999999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.96099999999999997,"15min":0.97099999999999997},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":1.006},"last":0.623},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95199999999999996,"5min":1.028,"15min":1.028},"last":0.48099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.219},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":1.018},"last":0.316},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.88500000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.435,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.90800000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.76800000000000002,"5min":0.82299999999999995,"15min":0.83599999999999997},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.41899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":1.0740000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.503,"5min":0.42999999999999999,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.96799999999999997},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.1839999999999999},"last":0.625},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.66000000000000003},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.433},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.70499999999999996,"15min":0.66800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48299999999999998,"15min":0.41099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41099999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.45000000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.224},"last":0.65700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575047,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1286144,"data_stored":920047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.83599999999999997},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.20300000000000001},"max":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":1.077},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58799999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59299999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.28699999999999998,"15min":0.221},"max":{"1min":0.64500000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.247},"max":{"1min":0.625,"5min":0.74099999999999999,"15min":0.745},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.47299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.02,"15min":1.1499999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93600000000000005},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.99099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.40200000000000002},"max":{"1min":0.90100000000000002,"5min":0.96299999999999997,"15min":1.1919999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":1.0029999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.1910000000000001},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.57999999999999996,"5min":0.435,"15min":0.39800000000000002},"max":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":1.0289999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0960000000000001},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:52:58.566 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.566 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.567 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.567 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.568 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.568 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.568 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.569 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-19T10:36:49.152681+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=49, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.569 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-19T10:36:42.521118+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=36, tm_sec=42, tm_wday=3, tm_yday=292, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=19, tm_hour=10, tm_min=52, tm_sec=53, tm_wday=3, tm_yday=292, tm_isdst=0) 2023-10-19T10:52:58.569 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-19T10:52:59.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:52:59.038 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:53:18.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-19T10:53:18.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.688+0000 7fce4112e640 1 Processor -- start 2023-10-19T10:53:18.691 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.688+0000 7fce4112e640 1 -- start start 2023-10-19T10:53:18.695 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.688+0000 7fce4112e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce3c06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce4112e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c0ca480 con 0x7fce3c0cd7d0 2023-10-19T10:53:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce4112e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c159990 con 0x7fce3c159280 2023-10-19T10:53:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce4112e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15d500 con 0x7fce3c15cd90 2023-10-19T10:53:18.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce33fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fce3c0cd7d0 0x7fce3c159e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33828/0 (socket says 172.21.15.50:33828) 2023-10-19T10:53:18.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce33fff640 1 -- 172.21.15.50:0/3160278498 learned_addr learned my addr 172.21.15.50:0/3160278498 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:53:18.697 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3b59e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce3c06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305428953 0 0) 0x7fce3c0ca480 con 0x7fce3c0cd7d0 2023-10-19T10:53:18.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce28003520 con 0x7fce3c0cd7d0 2023-10-19T10:53:18.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193092653 0 0) 0x7fce3c159990 con 0x7fce3c159280 2023-10-19T10:53:18.698 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c0ca480 con 0x7fce3c159280 2023-10-19T10:53:18.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776539247 0 0) 0x7fce28003520 con 0x7fce3c0cd7d0 2023-10-19T10:53:18.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 msgr2=0x7fce3c06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:18.699 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce3c06b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 >> v1:172.21.15.183:6789/0 conn(0x7fce3c159280 legacy=0x7fce3c0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c15d7e0 con 0x7fce3c0cd7d0 2023-10-19T10:53:18.700 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7fce24001670 con 0x7fce3c0cd7d0 2023-10-19T10:53:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3b59e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce3c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:53:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 >> v1:172.21.15.50:6789/0 conn(0x7fce3c0cd7d0 legacy=0x7fce3c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.701 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce28003b30 0x7fce28003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce28004d00 0x7fce28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15d500 con 0x7fce28004d00 2023-10-19T10:53:18.702 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2800ba30 con 0x7fce28003b30 2023-10-19T10:53:18.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.689+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2800bb70 con 0x7fce3c15cd90 2023-10-19T10:53:18.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3b59e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce28004d00 0x7fce28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.703 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3ad9d640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce28003b30 0x7fce28003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce33fff640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3b59e640 1 -- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 msgr2=0x7fce280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:18.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3b59e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15cd90 0x7fce280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.704 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3b59e640 1 -- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce28003b30 msgr2=0x7fce28003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3b59e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce28003b30 0x7fce28003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.690+0000 7fce3b59e640 1 -- 172.21.15.50:0/3160278498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009a70 con 0x7fce28004d00 2023-10-19T10:53:18.705 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce3b59e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce28004d00 0x7fce28005180 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e380 tx=0x7fce280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6175d6ae78ecc482 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3402cd70 con 0x7fce28004d00 2023-10-19T10:53:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400f040 con 0x7fce28004d00 2023-10-19T10:53:18.706 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce3a59c640 1 -- 172.21.15.50:0/3160278498 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34034640 con 0x7fce28004d00 2023-10-19T10:53:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce4112e640 1 -- 172.21.15.50:0/3160278498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce28004d00 msgr2=0x7fce28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce4112e640 1 --2- 172.21.15.50:0/3160278498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce28004d00 0x7fce28005180 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e380 tx=0x7fce280057f0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce3b59e640 1 -- 172.21.15.50:0/3160278498 reap_dead start 2023-10-19T10:53:18.707 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce4112e640 1 -- 172.21.15.50:0/3160278498 shutdown_connections 2023-10-19T10:53:18.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce4112e640 1 -- 172.21.15.50:0/3160278498 >> 172.21.15.50:0/3160278498 conn(0x7fce3c069e90 msgr2=0x7fce3c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:53:18.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce4112e640 1 -- 172.21.15.50:0/3160278498 shutdown_connections 2023-10-19T10:53:18.708 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.691+0000 7fce4112e640 1 -- 172.21.15.50:0/3160278498 wait complete. 2023-10-19T10:53:18.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.693+0000 7fce4112e640 1 Processor -- start 2023-10-19T10:53:18.709 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.693+0000 7fce4112e640 1 -- start start 2023-10-19T10:53:18.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce4112e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15b1f0 0x7fce3c0ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce4112e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 0x7fce3c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce4112e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3c154600 0x7fce3c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.712 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce4112e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0ac040 con 0x7fce3c15b5d0 2023-10-19T10:53:18.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce4112e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c154a80 con 0x7fce3c154600 2023-10-19T10:53:18.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce4112e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0c7bf0 con 0x7fce3c15b1f0 2023-10-19T10:53:18.713 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 0x7fce3c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 0x7fce3c1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59922/0 (socket says 172.21.15.50:59922) 2023-10-19T10:53:18.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 -- 172.21.15.50:0/943459796 learned_addr learned my addr 172.21.15.50:0/943459796 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:53:18.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce3ad9d640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15b1f0 0x7fce3c0ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.714 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce3b59e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3c154600 0x7fce3c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 -- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15b1f0 msgr2=0x7fce3c0ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15b1f0 0x7fce3c0ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 -- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3c154600 msgr2=0x7fce3c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.715 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3c154600 0x7fce3c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 -- 172.21.15.50:0/943459796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0c7ed0 con 0x7fce3c15b5d0 2023-10-19T10:53:18.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce33fff640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 0x7fce3c1540c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fce240072c0 tx=0x7fce240076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:18.716 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.694+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400f960 con 0x7fce3c15b5d0 2023-10-19T10:53:18.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.695+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400fae0 con 0x7fce3c15b5d0 2023-10-19T10:53:18.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.695+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c0ab980 con 0x7fce3c15b5d0 2023-10-19T10:53:18.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.695+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400fe90 con 0x7fce3c15b5d0 2023-10-19T10:53:18.717 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.695+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c0ac9a0 con 0x7fce3c15b5d0 2023-10-19T10:53:18.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.696+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c0c19b0 con 0x7fce3c15b5d0 2023-10-19T10:53:18.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.696+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fce2401d460 con 0x7fce3c15b5d0 2023-10-19T10:53:18.718 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.696+0000 7fce337fe640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fce1c06a050 0x7fce1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:18.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.697+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fce24088450 con 0x7fce3c15b5d0 2023-10-19T10:53:18.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.697+0000 7fce3ad9d640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fce1c06a050 0x7fce1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:18.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.697+0000 7fce3ad9d640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fce1c06a050 0x7fce1c06c510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0acf20 tx=0x7fce2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:18.719 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.701+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7fce24053d60 con 0x7fce3c15b5d0 2023-10-19T10:53:18.875 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.872+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 --> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce3c15d1e0 con 0x7fce1c06a050 2023-10-19T10:53:18.878 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.875+0000 7fce337fe640 1 -- 172.21.15.50:0/943459796 <== mgr.4099 v2:172.21.15.183:6832/645958316 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73168 (secure 0 0 0) 0x7fce2c0128c0 con 0x7fce1c06a050 2023-10-19T10:53:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:53:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2023-10-19T10:53:18.881 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.877+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fce1c06a050 msgr2=0x7fce1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.877+0000 7fce4112e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fce1c06a050 0x7fce1c06c510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0acf20 tx=0x7fce2c009040 comp rx=0 tx=0).stop 2023-10-19T10:53:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.877+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 msgr2=0x7fce3c1540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:18.882 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.877+0000 7fce4112e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 0x7fce3c1540c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fce240072c0 tx=0x7fce240076c0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.877+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 shutdown_connections 2023-10-19T10:53:18.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7fce1c06a050 0x7fce1c06c510 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3c154600 0x7fce3c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.883 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15b5d0 0x7fce3c1540c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 --2- 172.21.15.50:0/943459796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fce3c15b1f0 0x7fce3c0ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:18.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 >> 172.21.15.50:0/943459796 conn(0x7fce3c069e90 msgr2=0x7fce3c0adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:53:18.884 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 shutdown_connections 2023-10-19T10:53:18.885 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.878+0000 7fce4112e640 1 -- 172.21.15.50:0/943459796 wait complete. 2023-10-19T10:53:18.891 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2023-10-19T10:53:18.730819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239584,"kb_used_data":6552,"kb_used_omap":17,"kb_used_meta":232942,"kb_avail":754735136,"statfs":{"total":773094113280,"available":772848779264,"internally_reserved":0,"allocated":6709248,"data_stored":3813600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17557,"internal_metadata":238533483},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002372"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:53:02.397891+0000","last_change":"2023-10-19T10:53:02.397942+0000","last_active":"2023-10-19T10:53:02.397891+0000","last_peered":"2023-10-19T10:53:02.397891+0000","last_clean":"2023-10-19T10:53:02.397891+0000","last_became_active":"2023-10-19T10:36:50.169728+0000","last_became_peered":"2023-10-19T10:36:50.169728+0000","last_unstale":"2023-10-19T10:53:02.397891+0000","last_undegraded":"2023-10-19T10:53:02.397891+0000","last_fullsized":"2023-10-19T10:53:02.397891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:53:02.397797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:53:02.397797+0000","last_clean_scrub_stamp":"2023-10-19T10:53:02.397797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:12:33.059533+0000","scrub_duration":0.0015327940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:57.611031+0000","last_change":"2023-10-19T10:52:57.611064+0000","last_active":"2023-10-19T10:52:57.611031+0000","last_peered":"2023-10-19T10:52:57.611031+0000","last_clean":"2023-10-19T10:52:57.611031+0000","last_became_active":"2023-10-19T10:36:50.169530+0000","last_became_peered":"2023-10-19T10:36:50.169530+0000","last_unstale":"2023-10-19T10:52:57.611031+0000","last_undegraded":"2023-10-19T10:52:57.611031+0000","last_fullsized":"2023-10-19T10:52:57.611031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:52:57.610960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:52:57.610960+0000","last_clean_scrub_stamp":"2023-10-19T10:52:57.610960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:01:30.186414+0000","scrub_duration":0.0015945499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":103,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:53:02.564666+0000","last_change":"2023-10-19T10:53:02.564708+0000","last_active":"2023-10-19T10:53:02.564666+0000","last_peered":"2023-10-19T10:53:02.564666+0000","last_clean":"2023-10-19T10:53:02.564666+0000","last_became_active":"2023-10-19T10:36:50.494520+0000","last_became_peered":"2023-10-19T10:36:50.494520+0000","last_unstale":"2023-10-19T10:53:02.564666+0000","last_undegraded":"2023-10-19T10:53:02.564666+0000","last_fullsized":"2023-10-19T10:53:02.564666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:53:02.564591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:53:02.564591+0000","last_clean_scrub_stamp":"2023-10-19T10:53:02.564591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T19:37:12.211844+0000","scrub_duration":0.002305353,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:56.644687+0000","last_change":"2023-10-19T10:52:56.644722+0000","last_active":"2023-10-19T10:52:56.644687+0000","last_peered":"2023-10-19T10:52:56.644687+0000","last_clean":"2023-10-19T10:52:56.644687+0000","last_became_active":"2023-10-19T10:36:50.494281+0000","last_became_peered":"2023-10-19T10:36:50.494281+0000","last_unstale":"2023-10-19T10:52:56.644687+0000","last_undegraded":"2023-10-19T10:52:56.644687+0000","last_fullsized":"2023-10-19T10:52:56.644687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:52:56.644606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:52:56.644606+0000","last_clean_scrub_stamp":"2023-10-19T10:52:56.644606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T12:59:23.404082+0000","scrub_duration":0.0012398789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:53:03.738879+0000","last_change":"2023-10-19T10:53:03.738934+0000","last_active":"2023-10-19T10:53:03.738879+0000","last_peered":"2023-10-19T10:53:03.738879+0000","last_clean":"2023-10-19T10:53:03.738879+0000","last_became_active":"2023-10-19T10:36:50.167293+0000","last_became_peered":"2023-10-19T10:36:50.167293+0000","last_unstale":"2023-10-19T10:53:03.738879+0000","last_undegraded":"2023-10-19T10:53:03.738879+0000","last_fullsized":"2023-10-19T10:53:03.738879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:53:03.738763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:53:03.738763+0000","last_clean_scrub_stamp":"2023-10-19T10:53:03.738763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:26:48.741364+0000","scrub_duration":0.0020169459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:52:59.325545+0000","last_change":"2023-10-19T10:52:59.325597+0000","last_active":"2023-10-19T10:52:59.325545+0000","last_peered":"2023-10-19T10:52:59.325545+0000","last_clean":"2023-10-19T10:52:59.325545+0000","last_became_active":"2023-10-19T10:46:26.328144+0000","last_became_peered":"2023-10-19T10:46:26.328144+0000","last_unstale":"2023-10-19T10:52:59.325545+0000","last_undegraded":"2023-10-19T10:52:59.325545+0000","last_fullsized":"2023-10-19T10:52:59.325545+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:52:59.325478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:52:59.325478+0000","last_clean_scrub_stamp":"2023-10-19T10:52:59.325478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T20:18:56.752331+0000","scrub_duration":0.001220729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:53:04.581786+0000","last_change":"2023-10-19T10:53:04.581831+0000","last_active":"2023-10-19T10:53:04.581786+0000","last_peered":"2023-10-19T10:53:04.581786+0000","last_clean":"2023-10-19T10:53:04.581786+0000","last_became_active":"2023-10-19T10:36:50.169316+0000","last_became_peered":"2023-10-19T10:36:50.169316+0000","last_unstale":"2023-10-19T10:53:04.581786+0000","last_undegraded":"2023-10-19T10:53:04.581786+0000","last_fullsized":"2023-10-19T10:53:04.581786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:53:04.581695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:53:04.581695+0000","last_clean_scrub_stamp":"2023-10-19T10:53:04.581695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T11:29:18.342185+0000","scrub_duration":0.0022072250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":115,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:53:06.729085+0000","last_change":"2023-10-19T10:53:06.729135+0000","last_active":"2023-10-19T10:53:06.729085+0000","last_peered":"2023-10-19T10:53:06.729085+0000","last_clean":"2023-10-19T10:53:06.729085+0000","last_became_active":"2023-10-19T10:46:26.328870+0000","last_became_peered":"2023-10-19T10:46:26.328870+0000","last_unstale":"2023-10-19T10:53:06.729085+0000","last_undegraded":"2023-10-19T10:53:06.729085+0000","last_fullsized":"2023-10-19T10:53:06.729085+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-19T10:53:06.729005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-19T10:53:06.729005+0000","last_clean_scrub_stamp":"2023-10-19T10:53:06.729005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T14:04:35.761281+0000","scrub_duration":0.0022794989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"58'119","reported_seq":262,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-19T10:53:00.612918+0000","last_change":"2023-10-19T10:53:00.612985+0000","last_active":"2023-10-19T10:53:00.612918+0000","last_peered":"2023-10-19T10:53:00.612918+0000","last_clean":"2023-10-19T10:53:00.612918+0000","last_became_active":"2023-10-19T10:36:43.650032+0000","last_became_peered":"2023-10-19T10:36:43.650032+0000","last_unstale":"2023-10-19T10:53:00.612918+0000","last_undegraded":"2023-10-19T10:53:00.612918+0000","last_fullsized":"2023-10-19T10:53:00.612918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'119","last_scrub_stamp":"2023-10-19T10:53:00.612815+0000","last_deep_scrub":"58'119","last_deep_scrub_stamp":"2023-10-19T10:53:00.612815+0000","last_clean_scrub_stamp":"2023-10-19T10:53:00.612815+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T17:32:12.278103+0000","scrub_duration":0.0051456050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542347,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28268,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343572,"statfs":{"total":96636764160,"available":96607817728,"internally_reserved":0,"allocated":1282048,"data_stored":918905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.30599999999999999},"max":{"1min":0.72699999999999998,"5min":0.88400000000000001,"15min":0.91300000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.84599999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.629,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69199999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39800000000000002},"max":{"1min":0.748,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Oct 19 10:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.54100000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73499999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73599999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Oct 19 10:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.42899999999999999,"15min":0.374},"max":{"1min":0.621,"5min":0.80300000000000005,"15min":0.88},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.47899999999999998,"5min":0.45300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.81499999999999995,"15min":0.86399999999999999},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Oct 19 10:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.28699999999999998},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.749},"last":0.628},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48899999999999999,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.28999999999999998},"max":{"1min":0.67500000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48899999999999999,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32100000000000001,"15min":0.28000000000000003},"max":{"1min":0.72499999999999998,"5min":0.82499999999999996,"15min":0.84199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.30599999999999999},"max":{"1min":0.81799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Oct 19 10:53:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.35499999999999998},"max":{"1min":0.86199999999999999,"5min":0.872,"15min":0.872},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.33900000000000002,"15min":0.33600000000000002},"max":{"1min":0.70399999999999996,"5min":0.78100000000000003,"15min":0.79500000000000004},"last":0.66600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575050,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.39000000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.89900000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.41599999999999998,"5min":0.374,"15min":0.35399999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.38400000000000001},"max":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":0.83499999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.64300000000000002,"15min":0.628},"min":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.378},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Oct 19 10:52:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":1.042,"15min":1.042},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.629}]},{"osd":3,"last update":"Thu Oct 19 10:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.34499999999999997},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.075},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.432},"max":{"1min":0.79600000000000004,"5min":0.995,"15min":0.995},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Oct 19 10:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48399999999999999,"15min":0.46700000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.29499999999999998},"max":{"1min":0.76100000000000001,"5min":1.111,"15min":1.111},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73899999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Oct 19 10:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.502},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.30599999999999999},"max":{"1min":0.78000000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.40300000000000002,"5min":0.38100000000000001,"15min":0.33000000000000002},"max":{"1min":0.84699999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Oct 19 10:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.374,"5min":0.317,"15min":0.317},"max":{"1min":0.81799999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34399999999999997},"max":{"1min":0.87,"5min":0.92600000000000005,"15min":1.02},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:53:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.312},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Oct 19 10:53:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.32000000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93600000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.33400000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.84499999999999997},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Oct 19 10:52:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.752,"5min":0.752,"15min":0.79500000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Oct 19 10:53:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63800000000000001,"15min":0.623},"min":{"1min":0.53200000000000003,"5min":0.40000000000000002,"15min":0.34599999999999997},"max":{"1min":0.84799999999999998,"5min":0.85599999999999998,"15min":0.91300000000000003},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.33800000000000002},"max":{"1min":0.79200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Oct 19 10:53:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.307},"max":{"1min":0.70299999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.34300000000000003,"5min":0.29599999999999999,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:53:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.499,"15min":0.48599999999999999},"min":{"1min":0.39000000000000001,"5min":0.30499999999999999,"15min":0.30099999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.28299999999999997},"max":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.80000000000000004},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Oct 19 10:53:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.372,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.53700000000000003},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.65200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80200000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.69299999999999995,"5min":0.76400000000000001,"15min":0.79700000000000004},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Oct 19 10:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":0.83299999999999996,"15min":0.89700000000000002},"last":0.622},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83299999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71899999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Oct 19 10:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77100000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.749,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Oct 19 10:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44900000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.879},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.81499999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Oct 19 10:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46600000000000003,"15min":0.47899999999999998},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.29099999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.94799999999999995},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.313},"max":{"1min":0.83499999999999996,"5min":0.91100000000000003,"15min":0.97099999999999997},"last":0.45100000000000001}]},{"osd":7,"last update":"Thu Oct 19 10:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.50700000000000001},"min":{"1min":0.371,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.92600000000000005},"last":0.40799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":328537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.182},"max":{"1min":0.57099999999999995,"5min":0.61799999999999999,"15min":0.69699999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Oct 19 10:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.41199999999999998,"15min":0.40200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.24299999999999999},"max":{"1min":0.59999999999999998,"5min":0.67800000000000005,"15min":0.72199999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.442,"15min":0.435},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.222},"max":{"1min":0.53400000000000003,"5min":0.69399999999999995,"15min":0.74099999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46000000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.69999999999999996},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.36099999999999999},"max":{"1min":0.89100000000000001,"5min":0.92700000000000005,"15min":1.0149999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.376},"max":{"1min":0.875,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.35699999999999998},"max":{"1min":0.92900000000000005,"5min":1.006,"15min":1.006},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.47299999999999998,"5min":0.40400000000000003,"15min":0.39000000000000001},"max":{"1min":0.878,"5min":0.93700000000000006,"15min":0.997},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Oct 19 10:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.93000000000000005,"15min":0.96099999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":1.0349999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Oct 19 10:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.67600000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94899999999999995,"5min":0.96899999999999997,"15min":0.98699999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.42199999999999999},"max":{"1min":0.82399999999999995,"5min":0.96899999999999997,"15min":0.97199999999999998},"last":0.61199999999999999}]}]},{"osd":2,"up_from":54,"seq":231928234067,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":676,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5090,"internal_metadata":32369694},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.376,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52300000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.435,"5min":0.432,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":0.65500000000000003,"15min":0.70499999999999996},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46700000000000003,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":1.006,"15min":1.006},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.67900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79700000000000004,"5min":1.05,"15min":1.05},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":0.93999999999999995},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.96099999999999997,"15min":0.97099999999999997},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Oct 19 10:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":1.006},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95199999999999996,"5min":1.028,"15min":1.028},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":692224,"data_stored":329679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.219},"max":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":1.018},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.88500000000000001},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Oct 19 10:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46500000000000002,"15min":0.44600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.435,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.90800000000000003},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":0.76800000000000002,"5min":0.82299999999999995,"15min":0.83599999999999997},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.41899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":1.0740000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.503,"5min":0.42999999999999999,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.95299999999999996,"15min":0.96799999999999997},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.1839999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.66000000000000003},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.40500000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.66500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.433},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.70499999999999996,"15min":0.66800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48299999999999998,"15min":0.41099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.623}]},{"osd":7,"last update":"Thu Oct 19 10:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41099999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.45000000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.224},"last":0.84399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575051,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1286144,"data_stored":920047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.621,"5min":0.67300000000000004,"15min":0.83599999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.20300000000000001},"max":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":1.077},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Oct 19 10:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58799999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59299999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.28699999999999998,"15min":0.221},"max":{"1min":0.64500000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47399999999999998,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.247},"max":{"1min":0.625,"5min":0.74099999999999999,"15min":0.745},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.47299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.02,"15min":1.1499999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93600000000000005},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.48799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.99099999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.40200000000000002},"max":{"1min":0.90100000000000002,"5min":0.96299999999999997,"15min":1.1919999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":1.0029999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.1910000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Oct 19 10:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.57999999999999996,"5min":0.435,"15min":0.39800000000000002},"max":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":1.0289999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0960000000000001},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-19T10:53:18.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-19T10:53:19.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.998+0000 7f7527311640 1 Processor -- start 2023-10-19T10:53:19.001 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.998+0000 7f7527311640 1 -- start start 2023-10-19T10:53:19.003 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7527311640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7520071130 0x7f752006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7527311640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f752006bdf0 0x7f75200cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7527311640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75200cc0f0 con 0x7f7520158a10 2023-10-19T10:53:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7527311640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7520158df0 con 0x7f7520071130 2023-10-19T10:53:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7527311640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7520070d50 con 0x7f752006bdf0 2023-10-19T10:53:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75250ad640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f7520158a10 0x7f7520070540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33844/0 (socket says 172.21.15.50:33844) 2023-10-19T10:53:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75250ad640 1 -- 172.21.15.50:0/3891772218 learned_addr learned my addr 172.21.15.50:0/3891772218 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:53:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75248ac640 1 --2- 172.21.15.50:0/3891772218 >> v2:172.21.15.183:3300/0 conn(0x7f7520071130 0x7f752006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75258ae640 1 --2- 172.21.15.50:0/3891772218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f752006bdf0 0x7f75200cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7517fff640 1 -- 172.21.15.50:0/3891772218 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122539366 0 0) 0x7f75200cc0f0 con 0x7f7520158a10 2023-10-19T10:53:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f7517fff640 1 -- 172.21.15.50:0/3891772218 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f750c003520 con 0x7f7520158a10 2023-10-19T10:53:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75258ae640 1 -- 172.21.15.50:0/3891772218 >> v2:172.21.15.183:3300/0 conn(0x7f7520071130 msgr2=0x7f752006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75258ae640 1 --2- 172.21.15.50:0/3891772218 >> v2:172.21.15.183:3300/0 conn(0x7f7520071130 0x7f752006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75258ae640 1 -- 172.21.15.50:0/3891772218 >> v1:172.21.15.50:6789/0 conn(0x7f7520158a10 legacy=0x7f7520070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75258ae640 1 -- 172.21.15.50:0/3891772218 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200cc340 con 0x7f752006bdf0 2023-10-19T10:53:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:18.999+0000 7f75248ac640 1 --2- 172.21.15.50:0/3891772218 >> v2:172.21.15.183:3300/0 conn(0x7f7520071130 0x7f752006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-19T10:53:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.000+0000 7f75258ae640 1 --2- 172.21.15.50:0/3891772218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f752006bdf0 0x7f75200cce20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7518009d90 tx=0x7f751802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef23fc698a94ba6e server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7517fff640 1 -- 172.21.15.50:0/3891772218 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751802cbc0 con 0x7f752006bdf0 2023-10-19T10:53:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7517fff640 1 -- 172.21.15.50:0/3891772218 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751802cd20 con 0x7f752006bdf0 2023-10-19T10:53:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7517fff640 1 -- 172.21.15.50:0/3891772218 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518034660 con 0x7f752006bdf0 2023-10-19T10:53:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 -- 172.21.15.50:0/3891772218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f752006bdf0 msgr2=0x7f75200cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 --2- 172.21.15.50:0/3891772218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f752006bdf0 0x7f75200cce20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7518009d90 tx=0x7f751802be10 comp rx=0 tx=0).stop 2023-10-19T10:53:19.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 -- 172.21.15.50:0/3891772218 shutdown_connections 2023-10-19T10:53:19.012 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 --2- 172.21.15.50:0/3891772218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f752006bdf0 0x7f75200cce20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 --2- 172.21.15.50:0/3891772218 >> v2:172.21.15.183:3300/0 conn(0x7f7520071130 0x7f752006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 -- 172.21.15.50:0/3891772218 >> 172.21.15.50:0/3891772218 conn(0x7f75200afbe0 msgr2=0x7f75200affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:53:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 -- 172.21.15.50:0/3891772218 shutdown_connections 2023-10-19T10:53:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.001+0000 7f7527311640 1 -- 172.21.15.50:0/3891772218 wait complete. 2023-10-19T10:53:19.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.002+0000 7f7527311640 1 Processor -- start 2023-10-19T10:53:19.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.002+0000 7f7527311640 1 -- start start 2023-10-19T10:53:19.014 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f7527311640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 0x7f7520157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f7527311640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7520071130 0x7f75201f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f7527311640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7520158a10 0x7f75200c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f7527311640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7520157cd0 con 0x7f752006bdf0 2023-10-19T10:53:19.015 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f7527311640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75201f2900 con 0x7f7520158a10 2023-10-19T10:53:19.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f7527311640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75200accb0 con 0x7f7520071130 2023-10-19T10:53:19.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 0x7f7520157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.016 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 0x7f7520157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59936/0 (socket says 172.21.15.50:59936) 2023-10-19T10:53:19.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 -- 172.21.15.50:0/1469609383 learned_addr learned my addr 172.21.15.50:0/1469609383 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:53:19.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 -- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7520071130 msgr2=0x7f75201f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7520071130 0x7f75201f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.017 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 -- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7520158a10 msgr2=0x7f75200c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.003+0000 7f75250ad640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7520158a10 0x7f75200c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f75250ad640 1 -- 172.21.15.50:0/1469609383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7510009400 con 0x7f752006bdf0 2023-10-19T10:53:19.018 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f75250ad640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 0x7f7520157320 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c410 tx=0x7f751c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:19.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c005ce0 con 0x7f752006bdf0 2023-10-19T10:53:19.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c005e40 con 0x7f752006bdf0 2023-10-19T10:53:19.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7518009a70 con 0x7f752006bdf0 2023-10-19T10:53:19.019 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c0072f0 con 0x7f752006bdf0 2023-10-19T10:53:19.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.004+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75201f2590 con 0x7f752006bdf0 2023-10-19T10:53:19.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.005+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75201f2720 con 0x7f752006bdf0 2023-10-19T10:53:19.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.005+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f751c007510 con 0x7f752006bdf0 2023-10-19T10:53:19.020 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.006+0000 7f7515ffb640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.006+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f751c087a30 con 0x7f752006bdf0 2023-10-19T10:53:19.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.006+0000 7f75248ac640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.021 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.007+0000 7f75248ac640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7510006e80 tx=0x7f7510006dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:19.022 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.010+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f751c053400 con 0x7f752006bdf0 2023-10-19T10:53:19.188 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.185+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f752004e420 con 0x7f752006bdf0 2023-10-19T10:53:19.192 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.189+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f751c053400 con 0x7f752006bdf0 2023-10-19T10:53:19.193 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.189+0000 7f7515ffb640 1 -- 172.21.15.50:0/1469609383 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v17)=0 v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7f751c0170c0 con 0x7f752006bdf0 2023-10-19T10:53:19.195 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 msgr2=0x7f750406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7510006e80 tx=0x7f7510006dd0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 msgr2=0x7f7520157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 0x7f7520157320 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c410 tx=0x7f751c00c8d0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.196 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 shutdown_connections 2023-10-19T10:53:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f750406a050 0x7f750406c510 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7520158a10 0x7f75200c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7520071130 0x7f75201f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.197 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 --2- 172.21.15.50:0/1469609383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f752006bdf0 0x7f7520157320 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 >> 172.21.15.50:0/1469609383 conn(0x7f75200afbe0 msgr2=0x7f752006cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:53:19.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 shutdown_connections 2023-10-19T10:53:19.198 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.192+0000 7f7527311640 1 -- 172.21.15.50:0/1469609383 wait complete. 2023-10-19T10:53:19.206 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-19T10:53:19.206 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-19T10:53:19.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.310+0000 7f093f3f2640 1 Processor -- start 2023-10-19T10:53:19.313 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.310+0000 7f093f3f2640 1 -- start start 2023-10-19T10:53:19.317 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093f3f2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f093815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093f3f2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093806aa30 con 0x7f093815c570 2023-10-19T10:53:19.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093f3f2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093806b2d0 con 0x7f09380cd920 2023-10-19T10:53:19.318 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093f3f2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093815ce90 con 0x7f09380d0e30 2023-10-19T10:53:19.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093d18e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f093815c570 0x7f093806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:33848/0 (socket says 172.21.15.50:33848) 2023-10-19T10:53:19.319 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093d18e640 1 -- 172.21.15.50:0/168100225 learned_addr learned my addr 172.21.15.50:0/168100225 (peer_addr_for_me v1:172.21.15.50:0/0) 2023-10-19T10:53:19.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587242072 0 0) 0x7f093806aa30 con 0x7f093815c570 2023-10-19T10:53:19.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f093c98d640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f093815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.320 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0924003520 con 0x7f093815c570 2023-10-19T10:53:19.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850393838 0 0) 0x7f0924003520 con 0x7f093815c570 2023-10-19T10:53:19.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 msgr2=0x7f093815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:19.321 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f093815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 >> v1:172.21.15.183:6789/0 conn(0x7f09380cd920 legacy=0x7f09380d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380d1270 con 0x7f093815c570 2023-10-19T10:53:19.322 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1588095443 0 0) 0x7f09340017f0 con 0x7f093815c570 2023-10-19T10:53:19.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.311+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 >> v1:172.21.15.50:6789/0 conn(0x7f093815c570 legacy=0x7f093806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f092ffff640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f0924003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.323 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f092ffff640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09240047c0 0x7f0924004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f092ffff640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f09240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093815ce90 con 0x7f09240039a0 2023-10-19T10:53:19.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092400bab0 con 0x7f09240047c0 2023-10-19T10:53:19.324 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092400bc70 con 0x7f09380d0e30 2023-10-19T10:53:19.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d18e640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f0924003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093c98d640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09240047c0 0x7f0924004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.325 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d98f640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f09240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d18e640 1 -- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 msgr2=0x7f09240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:19.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d18e640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f09240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.326 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d18e640 1 -- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09240047c0 msgr2=0x7f0924004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-19T10:53:19.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d18e640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09240047c0 0x7f0924004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d18e640 1 -- 172.21.15.50:0/168100225 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0924003520 con 0x7f09240039a0 2023-10-19T10:53:19.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093c98d640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09240047c0 0x7f0924004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:53:19.327 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.312+0000 7f093d98f640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09380d0e30 0x7f09240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:53:19.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093d18e640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f0924003dc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f093400c190 tx=0x7f09240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3a2bc6c139463f server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:19.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934007270 con 0x7f09240039a0 2023-10-19T10:53:19.328 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f09340073f0 con 0x7f09240039a0 2023-10-19T10:53:19.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f092ffff640 1 -- 172.21.15.50:0/168100225 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093402c8a0 con 0x7f09240039a0 2023-10-19T10:53:19.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093f3f2640 1 -- 172.21.15.50:0/168100225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 msgr2=0x7f0924003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093f3f2640 1 --2- 172.21.15.50:0/168100225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f0924003dc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f093400c190 tx=0x7f09240057f0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.329 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093d98f640 1 -- 172.21.15.50:0/168100225 reap_dead start 2023-10-19T10:53:19.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093f3f2640 1 -- 172.21.15.50:0/168100225 shutdown_connections 2023-10-19T10:53:19.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093f3f2640 1 -- 172.21.15.50:0/168100225 >> 172.21.15.50:0/168100225 conn(0x7f0938089ff0 msgr2=0x7f093809b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:53:19.330 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.313+0000 7f093f3f2640 1 -- 172.21.15.50:0/168100225 shutdown_connections 2023-10-19T10:53:19.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.314+0000 7f093f3f2640 1 -- 172.21.15.50:0/168100225 wait complete. 2023-10-19T10:53:19.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 Processor -- start 2023-10-19T10:53:19.331 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 -- start start 2023-10-19T10:53:19.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f09380b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09380cbfd0 0x7f09380cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.334 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09381f1a60 0x7f09381f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380cb570 con 0x7f09240039a0 2023-10-19T10:53:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380cb7f0 con 0x7f09380cbfd0 2023-10-19T10:53:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093f3f2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09381f2520 con 0x7f09381f1a60 2023-10-19T10:53:19.335 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.315+0000 7f093d18e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f09380b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f09380b7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59954/0 (socket says 172.21.15.50:59954) 2023-10-19T10:53:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 -- 172.21.15.50:0/2348720196 learned_addr learned my addr 172.21.15.50:0/2348720196 (peer_addr_for_me v2:172.21.15.50:0/0) 2023-10-19T10:53:19.336 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093c98d640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09380cbfd0 0x7f09380cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d98f640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09381f1a60 0x7f09381f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 -- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09381f1a60 msgr2=0x7f09381f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.337 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09381f1a60 0x7f09381f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 -- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09380cbfd0 msgr2=0x7f09380cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09380cbfd0 0x7f09380cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 -- 172.21.15.50:0/2348720196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09381f2800 con 0x7f09240039a0 2023-10-19T10:53:19.338 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d98f640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09381f1a60 0x7f09381f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:53:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093c98d640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09380cbfd0 0x7f09380cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-19T10:53:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093d18e640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f09380b7b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0934001380 tx=0x7f0934001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:19.339 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934039070 con 0x7f09240039a0 2023-10-19T10:53:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f093402ccb0 con 0x7f09240039a0 2023-10-19T10:53:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09380cbab0 con 0x7f09240039a0 2023-10-19T10:53:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093403c040 con 0x7f09240039a0 2023-10-19T10:53:19.340 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.316+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09380c99a0 con 0x7f09240039a0 2023-10-19T10:53:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.317+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09380cbc40 con 0x7f09240039a0 2023-10-19T10:53:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.318+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0934032070 con 0x7f09240039a0 2023-10-19T10:53:19.341 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.318+0000 7f092dffb640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f091c06a080 0x7f091c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-19T10:53:19.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.318+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f09340a9790 con 0x7f09240039a0 2023-10-19T10:53:19.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.319+0000 7f093c98d640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f091c06a080 0x7f091c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-19T10:53:19.342 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.319+0000 7f093c98d640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f091c06a080 0x7f091c06c540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f09381f4fb0 tx=0x7f0928008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-19T10:53:19.343 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.323+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178480 (secure 0 0 0) 0x7f0934074f80 con 0x7f09240039a0 2023-10-19T10:53:19.537 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.534+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0938073990 con 0x7f09240039a0 2023-10-19T10:53:19.538 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.534+0000 7f092dffb640 1 -- 172.21.15.50:0/2348720196 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f09340785a0 con 0x7f09240039a0 2023-10-19T10:53:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-10-19T10:53:19.539 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6836/1755770253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1755770253},{"type":"v1","addr":"172.21.15.183:6836","nonce":1755770253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4104364230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4104364230},{"type":"v1","addr":"172.21.15.183:6837","nonce":4104364230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1519911167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":1519911167},{"type":"v1","addr":"172.21.15.50:6835","nonce":1519911167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":6423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/2144845391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2144845391},{"type":"v1","addr":"172.21.15.50:6834","nonce":2144845391}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-10-19T10:53:19.540 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 19 2023-10-19T10:53:19.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f091c06a080 msgr2=0x7f091c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.540 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f091c06a080 0x7f091c06c540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f09381f4fb0 tx=0x7f0928008040 comp rx=0 tx=0).stop 2023-10-19T10:53:19.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 msgr2=0x7f09380b7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-19T10:53:19.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f09380b7b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0934001380 tx=0x7f0934001b80 comp rx=0 tx=0).stop 2023-10-19T10:53:19.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 shutdown_connections 2023-10-19T10:53:19.541 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:6832/645958316,v1:172.21.15.183:6833/645958316] conn(0x7f091c06a080 0x7f091c06c540 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09381f1a60 0x7f09381f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09380cbfd0 0x7f09380cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.542 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 --2- 172.21.15.50:0/2348720196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09240039a0 0x7f09380b7b30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-19T10:53:19.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 >> 172.21.15.50:0/2348720196 conn(0x7f0938089ff0 msgr2=0x7f09380d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-19T10:53:19.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 shutdown_connections 2023-10-19T10:53:19.543 INFO:teuthology.orchestra.run.smithi050.stderr:2023-10-19T10:53:19.536+0000 7f093f3f2640 1 -- 172.21.15.50:0/2348720196 wait complete. 2023-10-19T10:53:19.550 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-19T10:53:19.550 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-19T10:53:19.551 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-19T10:53:19.551 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.560 INFO:tasks.ceph.mds.a:Stopped 2023-10-19T10:53:19.560 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-19T10:53:19.561 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.609 INFO:tasks.ceph.mds.c:Stopped 2023-10-19T10:53:19.610 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-19T10:53:19.610 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.622 INFO:tasks.ceph.mds.b:Stopped 2023-10-19T10:53:19.622 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-19T10:53:19.623 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.673 INFO:tasks.ceph.mds.d:Stopped 2023-10-19T10:53:19.673 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-19T10:53:19.674 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-19T10:53:19.674 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.700 INFO:tasks.ceph.osd.0:Stopped 2023-10-19T10:53:19.700 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-19T10:53:19.701 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.766 INFO:tasks.ceph.osd.1:Stopped 2023-10-19T10:53:19.767 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-19T10:53:19.767 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.832 INFO:tasks.ceph.osd.2:Stopped 2023-10-19T10:53:19.833 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-19T10:53:19.833 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.905 INFO:tasks.ceph.osd.3:Stopped 2023-10-19T10:53:19.906 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-19T10:53:19.906 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:19.938 INFO:tasks.ceph.osd.4:Stopped 2023-10-19T10:53:19.938 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-19T10:53:19.939 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.007 INFO:tasks.ceph.osd.5:Stopped 2023-10-19T10:53:20.008 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-19T10:53:20.008 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.076 INFO:tasks.ceph.osd.6:Stopped 2023-10-19T10:53:20.077 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-19T10:53:20.077 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.148 INFO:tasks.ceph.osd.7:Stopped 2023-10-19T10:53:20.149 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-19T10:53:20.149 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-19T10:53:20.149 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.172 INFO:tasks.ceph.mgr.y:Stopped 2023-10-19T10:53:20.172 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-19T10:53:20.173 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.218 INFO:tasks.ceph.mgr.x:Stopped 2023-10-19T10:53:20.219 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-19T10:53:20.219 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-19T10:53:20.220 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.231 INFO:tasks.ceph.mon.a:Stopped 2023-10-19T10:53:20.231 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-19T10:53:20.232 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.271 INFO:tasks.ceph.mon.b:Stopped 2023-10-19T10:53:20.272 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-19T10:53:20.272 INFO:teuthology.orchestra.run:waiting for 300 2023-10-19T10:53:20.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-10-19T10:53:20.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-19T10:53:20.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-19T10:53:20.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-19T10:53:20.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-19T10:53:20.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-19T10:53:20.325 INFO:tasks.ceph.mon.c:Stopped 2023-10-19T10:53:20.326 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-19T10:53:20.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 't responding to mclientcaps' | egrep -v 'file system flag refuse_client_session is set' | egrep -v 'Degraded data redundancy' | egrep -v MDS_CLIENTS_LAGGY | egrep -v 'Reduced data availability' | head -n 1 2023-10-19T10:53:20.366 INFO:teuthology.orchestra.run.smithi050.stdout:2023-10-19T10:36:55.296464+0000 mon.a (mon.0) 145 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-19T10:53:20.366 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-19T10:53:20.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:53:20.367 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-19T10:53:20.538 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:53:20.539 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-19T10:53:20.605 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:53:20.605 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-19T10:53:20.662 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:53:20.663 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-19T10:53:20.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:53:20.712 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-19T10:53:20.988 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:53:20.989 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-19T10:53:21.103 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:53:21.104 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-19T10:53:21.161 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:53:21.161 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-19T10:53:21.217 INFO:tasks.ceph:Archiving mon data... 2023-10-19T10:53:21.219 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/data/mon.a.tgz 2023-10-19T10:53:21.219 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-10-19T10:53:21.236 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.GN5PwEltWJ 2023-10-19T10:53:21.236 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.GN5PwEltWJ 2023-10-19T10:53:21.381 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.GN5PwEltWJ 2023-10-19T10:53:21.470 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.GN5PwEltWJ is 285KB 2023-10-19T10:53:21.566 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.GN5PwEltWJ 2023-10-19T10:53:21.581 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/data/mon.b.tgz 2023-10-19T10:53:21.581 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-10-19T10:53:21.599 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.BC9LUyLfl0 2023-10-19T10:53:21.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.BC9LUyLfl0 2023-10-19T10:53:21.745 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.BC9LUyLfl0 2023-10-19T10:53:21.830 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.BC9LUyLfl0 is 292KB 2023-10-19T10:53:21.925 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.BC9LUyLfl0 2023-10-19T10:53:21.941 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/data/mon.c.tgz 2023-10-19T10:53:21.941 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-10-19T10:53:21.995 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.ipZZ5sevvX 2023-10-19T10:53:21.996 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ipZZ5sevvX 2023-10-19T10:53:22.120 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.ipZZ5sevvX 2023-10-19T10:53:22.205 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.ipZZ5sevvX is 292KB 2023-10-19T10:53:22.302 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.ipZZ5sevvX 2023-10-19T10:53:22.316 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-19T10:53:22.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-19T10:53:22.319 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-19T10:53:22.382 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-19T10:53:22.382 DEBUG:teuthology.orchestra.run.smithi050:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-19T10:53:22.394 DEBUG:teuthology.orchestra.run.smithi183:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-19T10:53:22.429 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-19T10:53:22.450 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-19T10:53:22.455 INFO:tasks.ceph:Archiving crash dumps... 2023-10-19T10:53:22.457 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/remote/smithi050/crash 2023-10-19T10:53:22.458 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-19T10:53:22.506 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/remote/smithi183/crash 2023-10-19T10:53:22.507 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-19T10:53:22.534 INFO:tasks.ceph:Shutting down logrotate 2023-10-19T10:53:22.535 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:53:22.551 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-19T10:53:22.629 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-19T10:53:22.661 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-19T10:53:22.687 INFO:tasks.ceph:Compressing logs... 2023-10-19T10:53:22.688 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-19T10:53:22.703 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-19T10:53:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104570.log 2023-10-19T10:53:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110903.log 2023-10-19T10:53:22.732 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102980.log 2023-10-19T10:53:22.765 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-19T10:53:22.766 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88136.log 2023-10-19T10:53:22.766 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95663.log 2023-10-19T10:53:22.766 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88136.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-19T10:53:22.767 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.95663.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.88136.log.gz 2023-10-19T10:53:22.767 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-19T10:53:22.767 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-19T10:53:22.768 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88176.log 2023-10-19T10:53:22.768 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88296.log 2023-10-19T10:53:22.768 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88096.log 2023-10-19T10:53:22.768 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88296.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-19T10:53:22.769 INFO:teuthology.orchestra.run.smithi183.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88176.log.gz 2023-10-19T10:53:22.769 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88096.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88296.log.gz 2023-10-19T10:53:22.769 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-19T10:53:22.769 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88256.log 2023-10-19T10:53:22.770 INFO:teuthology.orchestra.run.smithi183.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88096.log.gz 2023-10-19T10:53:22.770 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88056.log 2023-10-19T10:53:22.770 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88256.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-19T10:53:22.771 INFO:teuthology.orchestra.run.smithi183.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88256.log.gz 2023-10-19T10:53:22.771 INFO:teuthology.orchestra.run.smithi183.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.1.95663.log.gz 2023-10-19T10:53:22.782 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-19T10:53:22.782 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88216.log 2023-10-19T10:53:22.782 INFO:teuthology.orchestra.run.smithi183.stderr: 88.2%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.88056.log.gz 2023-10-19T10:53:22.783 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94499.log 2023-10-19T10:53:22.783 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88216.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-19T10:53:22.783 INFO:teuthology.orchestra.run.smithi183.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88216.log.gz 2023-10-19T10:53:22.784 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.94499.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-19T10:53:22.784 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.log: 91.3% -- replaced with /var/log/ceph/ceph-client.1.94499.log.gz 2023-10-19T10:53:22.784 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94934.log 2023-10-19T10:53:22.784 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.b.log: 93.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-19T10:53:22.785 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-19T10:53:22.785 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.3.94934.log: 92.1% -- replaced with /var/log/ceph/ceph-client.3.94934.log.gz 2023-10-19T10:53:22.785 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88336.log 2023-10-19T10:53:22.785 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.d.log: 93.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-19T10:53:22.786 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-19T10:53:22.786 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.88336.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.88336.log.gz 2023-10-19T10:53:22.793 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.c.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-19T10:53:22.872 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102818.log 2023-10-19T10:53:22.873 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104570.log.gz 2023-10-19T10:53:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102980.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.110903.log.gz 2023-10-19T10:53:22.873 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.93614.log 2023-10-19T10:53:22.874 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102818.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.102980.log.gz 2023-10-19T10:53:22.874 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97093.log 2023-10-19T10:53:22.874 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1%/var/log/ceph/ceph-client.admin.93614.log: -- replaced with /var/log/ceph/ceph-client.admin.102818.log.gz 2023-10-19T10:53:22.874 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95875.log 2023-10-19T10:53:22.875 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93614.log.gz 2023-10-19T10:53:22.875 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105020.log 2023-10-19T10:53:22.875 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103232.log 2023-10-19T10:53:22.876 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97093.log.gz 2023-10-19T10:53:22.876 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105020.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-19T10:53:22.876 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95875.log.gz 2023-10-19T10:53:22.876 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105020.log.gz 2023-10-19T10:53:22.877 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97531.log 2023-10-19T10:53:22.877 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111072.log 2023-10-19T10:53:22.877 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103232.log.gz 2023-10-19T10:53:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93228.log 2023-10-19T10:53:22.878 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97531.log.gz 2023-10-19T10:53:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.111072.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.104886.log 2023-10-19T10:53:22.878 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97261.log 2023-10-19T10:53:22.879 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111072.log.gz 2023-10-19T10:53:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112026.log 2023-10-19T10:53:22.879 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.93228.log.gz 2023-10-19T10:53:22.879 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97261.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104886.log.gz 2023-10-19T10:53:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94016.log 2023-10-19T10:53:22.880 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97261.log.gz 2023-10-19T10:53:22.880 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.112026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93912.log 2023-10-19T10:53:22.881 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.94016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102650.log 2023-10-19T10:53:22.881 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112026.log.gz 2023-10-19T10:53:22.881 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93912.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94016.log.gz 2023-10-19T10:53:22.881 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97615.log 2023-10-19T10:53:22.882 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93912.log.gz 2023-10-19T10:53:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102182.log 2023-10-19T10:53:22.882 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109063.log 2023-10-19T10:53:22.883 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102650.log.gz 2023-10-19T10:53:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102182.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97615.log.gz 2023-10-19T10:53:22.883 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93826.log 2023-10-19T10:53:22.883 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102182.log.gz 2023-10-19T10:53:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104236.log 2023-10-19T10:53:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102140.log 2023-10-19T10:53:22.884 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109063.log.gz 2023-10-19T10:53:22.884 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104236.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93826.log.gz 2023-10-19T10:53:22.885 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93782.log 2023-10-19T10:53:22.885 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104236.log.gz 2023-10-19T10:53:22.885 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110717.log 2023-10-19T10:53:22.886 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110381.log 2023-10-19T10:53:22.886 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102140.log.gz 2023-10-19T10:53:22.886 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110717.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93782.log.gz 2023-10-19T10:53:22.886 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96832.log 2023-10-19T10:53:22.887 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110717.log.gz 2023-10-19T10:53:22.887 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110759.log 2023-10-19T10:53:22.887 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106759.log 2023-10-19T10:53:22.887 INFO:teuthology.orchestra.run.smithi050.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110381.log.gz 2023-10-19T10:53:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110759.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96832.log.gz 2023-10-19T10:53:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96557.log 2023-10-19T10:53:22.888 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110759.log.gz 2023-10-19T10:53:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.106759.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.106196.log 2023-10-19T10:53:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110591.log 2023-10-19T10:53:22.889 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106759.log.gz 2023-10-19T10:53:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106196.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96557.log.gz 2023-10-19T10:53:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101188.log 2023-10-19T10:53:22.890 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106196.log.gz 2023-10-19T10:53:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109189.log 2023-10-19T10:53:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101020.log 2023-10-19T10:53:22.891 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110591.log.gz 2023-10-19T10:53:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109189.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101188.log.gz 2023-10-19T10:53:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.98961.log 2023-10-19T10:53:22.891 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109189.log.gz 2023-10-19T10:53:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101678.log 2023-10-19T10:53:22.892 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.2.98961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103526.log 2023-10-19T10:53:22.892 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101020.log.gz 2023-10-19T10:53:22.893 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108606.log 2023-10-19T10:53:22.893 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101678.log.gz 2023-10-19T10:53:22.893 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104936.log 2023-10-19T10:53:22.893 INFO:teuthology.orchestra.run.smithi050.stderr: 92.1% 88.4% -- replaced with /var/log/ceph/ceph-client.2.98961.log.gz 2023-10-19T10:53:22.894 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103526.log.gz 2023-10-19T10:53:22.894 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109231.log 2023-10-19T10:53:22.894 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108606.log: /var/log/ceph/ceph-client.admin.104936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97783.log 2023-10-19T10:53:22.894 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108606.log.gz 2023-10-19T10:53:22.895 INFO:teuthology.orchestra.run.smithi050.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104936.log.gz 2023-10-19T10:53:22.895 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93972.log 2023-10-19T10:53:22.895 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108746.log 2023-10-19T10:53:22.896 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109231.log.gz 2023-10-19T10:53:22.896 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0%/var/log/ceph/ceph-client.admin.93972.log: -- replaced with /var/log/ceph/ceph-client.admin.97783.log.gz 2023-10-19T10:53:22.896 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103022.log 2023-10-19T10:53:22.896 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93972.log.gz 2023-10-19T10:53:22.897 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.108746.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.108114.log 2023-10-19T10:53:22.897 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96556.log 2023-10-19T10:53:22.897 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108746.log.gz 2023-10-19T10:53:22.897 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108114.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103022.log.gz 2023-10-19T10:53:22.898 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88006.log 2023-10-19T10:53:22.898 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108114.log.gz 2023-10-19T10:53:22.898 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.96556.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99287.log 2023-10-19T10:53:22.899 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.88006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100547.log 2023-10-19T10:53:22.899 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96556.log.gz 2023-10-19T10:53:22.899 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99287.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88006.log.gz 2023-10-19T10:53:22.900 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97135.log 2023-10-19T10:53:22.900 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99287.log.gz 2023-10-19T10:53:22.900 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104612.log 2023-10-19T10:53:22.900 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.97135.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.100673.log 2023-10-19T10:53:22.901 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100547.log.gz 2023-10-19T10:53:22.901 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104612.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97135.log.gz -5 2023-10-19T10:53:22.901 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.112266.log 2023-10-19T10:53:22.901 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104612.log.gz 2023-10-19T10:53:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103820.log 2023-10-19T10:53:22.902 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.112266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102566.log 2023-10-19T10:53:22.902 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100673.log.gz 2023-10-19T10:53:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103820.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112266.log.gz 2023-10-19T10:53:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101146.log 2023-10-19T10:53:22.903 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103820.log.gz 2023-10-19T10:53:22.903 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99749.log 2023-10-19T10:53:22.904 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110264.log 2023-10-19T10:53:22.904 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102566.log.gz 2023-10-19T10:53:22.904 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99749.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101146.log.gz 2023-10-19T10:53:22.905 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97573.log 2023-10-19T10:53:22.905 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99749.log.gz 2023-10-19T10:53:22.905 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101846.log 2023-10-19T10:53:22.905 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102298.log 2023-10-19T10:53:22.906 INFO:teuthology.orchestra.run.smithi050.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.110264.log.gz 2023-10-19T10:53:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101846.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97573.log.gz 2023-10-19T10:53:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102014.log 2023-10-19T10:53:22.907 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101846.log.gz 2023-10-19T10:53:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106154.log 2023-10-19T10:53:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102298.log: /var/log/ceph/ceph-client.admin.102014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100757.log 2023-10-19T10:53:22.907 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102298.log.gz 2023-10-19T10:53:22.908 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106154.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102014.log.gz 2023-10-19T10:53:22.908 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99371.log 2023-10-19T10:53:22.908 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106154.log.gz 2023-10-19T10:53:22.908 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106050.log 2023-10-19T10:53:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99371.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100757.log.gz -5 2023-10-19T10:53:22.909 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102466.log 2023-10-19T10:53:22.909 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99371.log.gz 2023-10-19T10:53:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102608.log 2023-10-19T10:53:22.910 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106050.log.gz 2023-10-19T10:53:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105274.log 2023-10-19T10:53:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100043.log 2023-10-19T10:53:22.911 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102466.log.gz 2023-10-19T10:53:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105274.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102608.log.gz 2023-10-19T10:53:22.911 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93698.log 2023-10-19T10:53:22.911 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105274.log.gz 2023-10-19T10:53:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102382.log 2023-10-19T10:53:22.912 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103274.log 2023-10-19T10:53:22.912 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100043.log.gz 2023-10-19T10:53:22.913 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102382.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93698.log.gz 2023-10-19T10:53:22.913 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104738.log 2023-10-19T10:53:22.913 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102382.log.gz 2023-10-19T10:53:22.913 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103610.log 2023-10-19T10:53:22.914 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96217.log 2023-10-19T10:53:22.914 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103274.log.gz 2023-10-19T10:53:22.914 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103610.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104738.log.gz 2023-10-19T10:53:22.914 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100337.log 2023-10-19T10:53:22.915 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103610.log.gz 2023-10-19T10:53:22.915 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99665.log 2023-10-19T10:53:22.915 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96560.log 2023-10-19T10:53:22.916 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96217.log.gz 2023-10-19T10:53:22.916 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99665.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100337.log.gz 2023-10-19T10:53:22.916 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109809.log 2023-10-19T10:53:22.916 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99665.log.gz 2023-10-19T10:53:22.917 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96559.log 2023-10-19T10:53:22.917 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110988.log 2023-10-19T10:53:22.917 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96560.log.gz 2023-10-19T10:53:22.917 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96559.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109809.log.gz 2023-10-19T10:53:22.918 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96836.log 2023-10-19T10:53:22.918 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96559.log.gz 2023-10-19T10:53:22.918 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100589.log 2023-10-19T10:53:22.919 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110988.log: /var/log/ceph/ceph-client.admin.96836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111522.log 2023-10-19T10:53:22.919 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110988.log.gz 2023-10-19T10:53:22.919 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100589.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96836.log.gz 2023-10-19T10:53:22.919 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96219.log 2023-10-19T10:53:22.920 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100589.log.gz 2023-10-19T10:53:22.920 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112226.log 2023-10-19T10:53:22.920 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97825.log 2023-10-19T10:53:22.920 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111522.log.gz 2023-10-19T10:53:22.921 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.112226.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96219.log.gz 2023-10-19T10:53:22.921 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112226.log.gz 2023-10-19T10:53:22.921 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97741.log 2023-10-19T10:53:22.921 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107035.log 2023-10-19T10:53:22.922 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97741.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97825.log.gz 2023-10-19T10:53:22.922 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111900.log 2023-10-19T10:53:22.922 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97741.log.gz 2023-10-19T10:53:22.923 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.107035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111858.log 2023-10-19T10:53:22.923 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107035.log.gz 2023-10-19T10:53:22.923 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95461.log 2023-10-19T10:53:22.923 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96224.log 2023-10-19T10:53:22.924 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111900.log.gz 2023-10-19T10:53:22.924 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95461.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111858.log.gz 2023-10-19T10:53:22.924 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104030.log 2023-10-19T10:53:22.925 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95461.log.gz 2023-10-19T10:53:22.925 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100253.log 2023-10-19T10:53:22.925 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94184.log 2023-10-19T10:53:22.925 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96224.log.gz 2023-10-19T10:53:22.926 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100253.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104030.log.gz 2023-10-19T10:53:22.926 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102692.log 2023-10-19T10:53:22.926 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100253.log.gz 2023-10-19T10:53:22.927 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.94184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102424.log 2023-10-19T10:53:22.927 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106112.log 2023-10-19T10:53:22.927 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94184.log.gz 2023-10-19T10:53:22.927 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102424.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102692.log.gz 2023-10-19T10:53:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95665.log 2023-10-19T10:53:22.928 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102424.log.gz 2023-10-19T10:53:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97219.log 2023-10-19T10:53:22.928 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103148.log 2023-10-19T10:53:22.929 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106112.log.gz 2023-10-19T10:53:22.929 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97219.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95665.log.gz 2023-10-19T10:53:22.929 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110180.log 2023-10-19T10:53:22.930 INFO:teuthology.orchestra.run.smithi050.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.97219.log.gz 2023-10-19T10:53:22.930 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109935.log 2023-10-19T10:53:22.930 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102056.log 2023-10-19T10:53:22.930 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103148.log.gz 2023-10-19T10:53:22.931 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109935.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110180.log.gz 2023-10-19T10:53:22.931 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95193.log 2023-10-19T10:53:22.931 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109935.log.gz 2023-10-19T10:53:22.931 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108312.log 2023-10-19T10:53:22.932 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93529.log 2023-10-19T10:53:22.932 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102056.log.gz 2023-10-19T10:53:22.932 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108312.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95193.log.gz 2023-10-19T10:53:22.933 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109568.log 2023-10-19T10:53:22.933 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108312.log.gz 2023-10-19T10:53:22.933 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93529.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-19T10:53:22.933 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.109568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112308.log 2023-10-19T10:53:22.934 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93529.log.gz 2023-10-19T10:53:22.934 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104978.log 2023-10-19T10:53:22.934 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.112308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93406.log 2023-10-19T10:53:22.935 INFO:teuthology.orchestra.run.smithi050.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112308.log.gz 2023-10-19T10:53:22.935 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110549.log 2023-10-19T10:53:22.935 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95959.log 2023-10-19T10:53:22.935 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93406.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104978.log.gz 2023-10-19T10:53:22.937 INFO:teuthology.orchestra.run.smithi050.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.109568.log.gz 2023-10-19T10:53:22.937 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96835.log 2023-10-19T10:53:22.937 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93406.log.gz 2023-10-19T10:53:22.938 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103106.log 2023-10-19T10:53:22.938 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110549.log.gz 2023-10-19T10:53:22.938 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-19T10:53:22.939 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105592.log 2023-10-19T10:53:22.939 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95959.log.gz 2023-10-19T10:53:22.939 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96835.log: /var/log/ceph/ceph-client.admin.103106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108607.log 2023-10-19T10:53:22.939 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96835.log.gz 2023-10-19T10:53:22.940 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95545.log 2023-10-19T10:53:22.940 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103106.log.gz 2023-10-19T10:53:22.940 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105592.log.gz 2023-10-19T10:53:22.940 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97489.log 2023-10-19T10:53:22.941 INFO:teuthology.orchestra.run.smithi050.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108607.log.gz 2023-10-19T10:53:22.941 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99581.log 2023-10-19T10:53:22.941 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97489.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95545.log.gz 2023-10-19T10:53:22.942 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102256.log 2023-10-19T10:53:22.942 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97489.log.gz 2023-10-19T10:53:22.942 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109105.log 2023-10-19T10:53:22.942 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96561.log 2023-10-19T10:53:22.943 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99581.log.gz 2023-10-19T10:53:22.943 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105484.log 2023-10-19T10:53:22.943 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102256.log.gz 2023-10-19T10:53:22.943 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109105.log.gz 2023-10-19T10:53:22.944 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95623.log 2023-10-19T10:53:22.944 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108468.log 2023-10-19T10:53:22.944 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96561.log.gz 2023-10-19T10:53:22.945 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.105484.log.gz 2023-10-19T10:53:22.945 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106440.log 2023-10-19T10:53:22.945 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104842.log 2023-10-19T10:53:22.945 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95623.log.gz 2023-10-19T10:53:22.946 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106440.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108468.log.gz 2023-10-19T10:53:22.946 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111480.log 2023-10-19T10:53:22.946 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106440.log.gz 2023-10-19T10:53:22.946 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104360.log 2023-10-19T10:53:22.947 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104842.log.gz 2023-10-19T10:53:22.947 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93870.log 2023-10-19T10:53:22.947 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95293.log 2023-10-19T10:53:22.948 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111480.log.gz 2023-10-19T10:53:22.948 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104360.log.gz 2023-10-19T10:53:22.948 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96223.log 2023-10-19T10:53:22.948 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93870.log.gz 2023-10-19T10:53:22.949 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96895.log 2023-10-19T10:53:22.949 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99875.log 2023-10-19T10:53:22.949 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95293.log.gz 2023-10-19T10:53:22.949 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96223.log.gz 2023-10-19T10:53:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93147.log 2023-10-19T10:53:22.950 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0%/var/log/ceph/ceph-client.admin.99875.log: -- replaced with /var/log/ceph/ceph-client.admin.96895.log.gz 2023-10-19T10:53:22.950 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95377.log 2023-10-19T10:53:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100463.log 2023-10-19T10:53:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93147.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99875.log.gz 2023-10-19T10:53:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95377.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93147.log.gz 2023-10-19T10:53:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96225.log 2023-10-19T10:53:22.952 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95377.log.gz 2023-10-19T10:53:22.952 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104402.log 2023-10-19T10:53:22.952 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106537.log 2023-10-19T10:53:22.952 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100463.log.gz 2023-10-19T10:53:22.953 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96225.log.gz 2023-10-19T10:53:22.953 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103442.log 2023-10-19T10:53:22.953 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104402.log.gz 2023-10-19T10:53:22.954 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.106537.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.108175.log 2023-10-19T10:53:22.954 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111396.log 2023-10-19T10:53:22.954 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106537.log.gz 2023-10-19T10:53:22.954 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108175.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103442.log.gz 2023-10-19T10:53:22.955 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99917.log 2023-10-19T10:53:22.955 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108175.log.gz 2023-10-19T10:53:22.955 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106617.log 2023-10-19T10:53:22.955 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111396.log: /var/log/ceph/ceph-client.admin.99917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105442.log 2023-10-19T10:53:22.956 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111396.log.gz 2023-10-19T10:53:22.956 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106617.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.99917.log.gz 2023-10-19T10:53:22.956 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.110843.log 2023-10-19T10:53:22.957 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101558.log 2023-10-19T10:53:22.957 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105442.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106617.log.gz 2023-10-19T10:53:22.957 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103988.log 2023-10-19T10:53:22.957 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105442.log.gz 2023-10-19T10:53:22.958 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101558.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110843.log.gz 2023-10-19T10:53:22.958 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105851.log 2023-10-19T10:53:22.958 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101558.log.gz 2023-10-19T10:53:22.958 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.103988.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98587.log 2023-10-19T10:53:22.959 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.105851.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103988.log.gz 2023-10-19T10:53:22.959 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95419.log 2023-10-19T10:53:22.959 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.98587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111156.log 2023-10-19T10:53:22.960 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103778.log 2023-10-19T10:53:22.960 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98587.log.gz 2023-10-19T10:53:22.960 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95419.log.gz 2023-10-19T10:53:22.960 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100421.log 2023-10-19T10:53:22.961 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111156.log.gz 2023-10-19T10:53:22.961 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100978.log 2023-10-19T10:53:22.961 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103778.log: /var/log/ceph/ceph-client.admin.100421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101972.log 2023-10-19T10:53:22.961 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103778.log.gz 2023-10-19T10:53:22.962 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100421.log.gz 2023-10-19T10:53:22.962 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108313.log 2023-10-19T10:53:22.962 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100978.log.gz 2023-10-19T10:53:22.963 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105358.log 2023-10-19T10:53:22.963 INFO:teuthology.orchestra.run.smithi050.stderr:gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.101972.log.gz 2023-10-19T10:53:22.963 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96001.log 2023-10-19T10:53:22.963 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108313.log: /var/log/ceph/ceph-client.admin.105358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109309.log 2023-10-19T10:53:22.964 INFO:teuthology.orchestra.run.smithi050.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108313.log.gz 2023-10-19T10:53:22.964 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96001.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105358.log.gz 2023-10-19T10:53:22.964 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104444.log 2023-10-19T10:53:22.965 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103736.log 2023-10-19T10:53:22.965 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109309.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96001.log.gz 2023-10-19T10:53:22.965 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110465.log 2023-10-19T10:53:22.965 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109309.log.gz 2023-10-19T10:53:22.966 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104444.log.gz 2023-10-19T10:53:22.966 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111984.log 2023-10-19T10:53:22.966 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97447.log 2023-10-19T10:53:22.966 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103736.log.gz 2023-10-19T10:53:22.967 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111984.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110465.log.gz 2023-10-19T10:53:22.967 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104156.log 2023-10-19T10:53:22.967 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97447.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111984.log.gz 2023-10-19T10:53:22.967 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112153.log 2023-10-19T10:53:22.968 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97447.log.gz 2023-10-19T10:53:22.968 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97321.log 2023-10-19T10:53:22.968 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104156.log: /var/log/ceph/ceph-client.admin.112153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95791.log 2023-10-19T10:53:22.969 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104156.log.gz 2023-10-19T10:53:22.969 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97321.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112153.log.gz 2023-10-19T10:53:22.969 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104528.log 2023-10-19T10:53:22.969 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103316.log 2023-10-19T10:53:22.970 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95791.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97321.log.gz 2023-10-19T10:53:22.970 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93326.log 2023-10-19T10:53:22.970 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95791.log.gz 2023-10-19T10:53:22.970 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104528.log.gzgzip 2023-10-19T10:53:22.971 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-19T10:53:22.971 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103316.log: /var/log/ceph/ceph-client.admin.93326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96543.log 2023-10-19T10:53:22.971 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103316.log.gz 2023-10-19T10:53:22.972 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93326.log.gz 2023-10-19T10:53:22.972 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103484.log 2023-10-19T10:53:22.972 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102734.log 2023-10-19T10:53:22.972 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96543.log: /var/log/ceph/ceph-client.admin.103484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99623.log 2023-10-19T10:53:22.973 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96543.log.gz 2023-10-19T10:53:22.973 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103484.log.gz 2023-10-19T10:53:22.973 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104072.log 2023-10-19T10:53:22.973 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100505.log 2023-10-19T10:53:22.974 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99623.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102734.log.gz 2023-10-19T10:53:22.974 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97657.log 2023-10-19T10:53:22.974 INFO:teuthology.orchestra.run.smithi050.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99623.log.gz 2023-10-19T10:53:22.975 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100505.log: gzip 88.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.104072.log.gz -- 2023-10-19T10:53:22.975 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.100799.log 2023-10-19T10:53:22.975 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96757.log 2023-10-19T10:53:22.975 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100505.log.gz 2023-10-19T10:53:22.976 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100799.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97657.log.gz 2023-10-19T10:53:22.976 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96897.log 2023-10-19T10:53:22.976 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96757.log: gzip -5 --verbose 88.3% -- -- replaced with /var/log/ceph/ceph-client.admin.100799.log.gz /var/log/ceph/ceph-client.admin.95025.log 2023-10-19T10:53:22.976 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:22.977 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96757.log.gz 2023-10-19T10:53:22.977 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101474.log 2023-10-19T10:53:22.977 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101230.log 2023-10-19T10:53:22.977 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95025.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96897.log.gz 2023-10-19T10:53:22.978 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99092.log 2023-10-19T10:53:22.978 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95025.log.gz 2023-10-19T10:53:22.978 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.101230.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.101516.log 2023-10-19T10:53:22.979 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101474.log.gz 2023-10-19T10:53:22.979 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95749.log 2023-10-19T10:53:22.979 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101230.log.gz 2023-10-19T10:53:22.979 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103064.log 2023-10-19T10:53:22.980 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99092.log.gz 2023-10-19T10:53:22.980 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106008.log 2023-10-19T10:53:22.980 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101516.log.gz 2023-10-19T10:53:22.981 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95749.log.gz 2023-10-19T10:53:22.981 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110633.log 2023-10-19T10:53:22.981 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104318.log 2023-10-19T10:53:22.981 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103064.log.gz 2023-10-19T10:53:22.982 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106008.log.gz 2023-10-19T10:53:22.982 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99329.log 2023-10-19T10:53:22.982 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100631.log 2023-10-19T10:53:22.982 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104318.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110633.log.gz 2023-10-19T10:53:22.983 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96896.log 2023-10-19T10:53:22.983 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104318.log.gz 2023-10-19T10:53:22.983 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99329.log.gz 2023-10-19T10:53:22.984 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100127.log 2023-10-19T10:53:22.984 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102508.log 2023-10-19T10:53:22.984 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96896.log: 96.5% -- replaced with /var/log/ceph/ceph-client.0.105851.log.gz 2023-10-19T10:53:22.984 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100631.log.gz 2023-10-19T10:53:22.985 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95503.log 2023-10-19T10:53:22.985 INFO:teuthology.orchestra.run.smithi050.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96896.log.gz 2023-10-19T10:53:22.985 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99245.log 2023-10-19T10:53:22.985 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100127.log.gz 2023-10-19T10:53:22.986 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102508.log.gz 2023-10-19T10:53:22.986 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104654.log 2023-10-19T10:53:22.986 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.99245.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.111774.log 2023-10-19T10:53:22.987 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95503.log.gz 2023-10-19T10:53:22.987 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104654.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99245.log.gz 2023-10-19T10:53:22.987 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111732.log 2023-10-19T10:53:22.987 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95833.log 2023-10-19T10:53:22.988 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111774.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104654.log.gz 2023-10-19T10:53:22.988 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100085.log 2023-10-19T10:53:22.988 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111774.log.gz 2023-10-19T10:53:22.989 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111732.log.gz 2023-10-19T10:53:22.989 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99135.log 2023-10-19T10:53:22.989 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95833.log: /var/log/ceph/ceph-client.admin.100085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107197.log 2023-10-19T10:53:22.989 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95833.log.gz 2023-10-19T10:53:22.990 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100085.log.gz 2023-10-19T10:53:22.990 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100169.log 2023-10-19T10:53:22.990 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111564.log 2023-10-19T10:53:22.991 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99135.log.gz 2023-10-19T10:53:22.991 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.107197.log: /var/log/ceph/ceph-client.admin.100169.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-19T10:53:22.991 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107197.log.gz 2023-10-19T10:53:22.991 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100169.log.gz 2023-10-19T10:53:22.992 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103694.log 2023-10-19T10:53:22.992 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111606.log 2023-10-19T10:53:22.992 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111564.log.gz 2023-10-19T10:53:22.992 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.103694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100001.log 2023-10-19T10:53:22.993 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111606.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103694.log.gz 2023-10-19T10:53:22.993 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103862.log 2023-10-19T10:53:22.993 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111606.log.gz 2023-10-19T10:53:22.994 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106896.log 2023-10-19T10:53:22.994 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102776.log 2023-10-19T10:53:22.994 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103862.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100001.log.gz 2023-10-19T10:53:22.994 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106896.log: gzip -5 --verbose 88.9% -- /var/log/ceph/ceph-client.admin.96553.log 2023-10-19T10:53:22.995 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103862.log.gz 2023-10-19T10:53:22.995 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102776.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106896.log.gz 2023-10-19T10:53:22.995 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98465.log 2023-10-19T10:53:22.995 INFO:teuthology.orchestra.run.smithi050.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102776.log.gz 2023-10-19T10:53:22.996 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100211.log 2023-10-19T10:53:22.996 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96553.log: /var/log/ceph/ceph-client.0.98465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95067.log 2023-10-19T10:53:22.996 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96553.log.gz 2023-10-19T10:53:22.997 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102098.log 2023-10-19T10:53:22.997 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95067.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100211.log.gz 2023-10-19T10:53:22.998 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.98465.log.gz 2023-10-19T10:53:22.998 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94800.log 2023-10-19T10:53:22.998 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95067.log.gz 2023-10-19T10:53:22.999 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108174.log 2023-10-19T10:53:22.999 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102098.log: /var/log/ceph/ceph-client.admin.94800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96837.log 2023-10-19T10:53:22.999 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102098.log.gz 2023-10-19T10:53:22.999 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100295.log 2023-10-19T10:53:23.000 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96837.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108174.log.gz 2023-10-19T10:53:23.000 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105526.log 2023-10-19T10:53:23.000 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100295.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96837.log.gz 2023-10-19T10:53:23.001 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103652.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100295.log.gz 2023-10-19T10:53:23.001 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.001 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108884.log 2023-10-19T10:53:23.001 INFO:teuthology.orchestra.run.smithi050.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.94800.log.gz 2023-10-19T10:53:23.002 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94983.log 2023-10-19T10:53:23.002 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108884.log: gzip -5 --verbose 90.9% -- replaced with /var/log/ceph/ceph-client.admin.105526.log.gz 2023-10-19T10:53:23.002 INFO:teuthology.orchestra.run.smithi050.stderr: -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103652.log.gz 2023-10-19T10:53:23.003 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.111312.log 2023-10-19T10:53:23.003 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.94983.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108884.log.gz 2023-10-19T10:53:23.003 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107036.log 2023-10-19T10:53:23.003 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94983.log.gz 2023-10-19T10:53:23.004 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87754.log 2023-10-19T10:53:23.004 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111312.log: /var/log/ceph/ceph-client.admin.107036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102340.log 2023-10-19T10:53:23.004 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111312.log.gz 2023-10-19T10:53:23.004 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94142.log 2023-10-19T10:53:23.005 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.87754.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.107036.log.gz 2023-10-19T10:53:23.005 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102340.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87754.log.gz -5 2023-10-19T10:53:23.005 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98079.log 2023-10-19T10:53:23.006 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102340.log.gz 2023-10-19T10:53:23.006 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.94142.log: --verbose -- /var/log/ceph/ceph-client.admin.110423.log 2023-10-19T10:53:23.006 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.98079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97177.log 2023-10-19T10:53:23.006 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94142.log.gz 2023-10-19T10:53:23.007 INFO:teuthology.orchestra.run.smithi050.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98079.log.gz 2023-10-19T10:53:23.007 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103946.log 2023-10-19T10:53:23.007 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93740.log 2023-10-19T10:53:23.008 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97177.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110423.log.gz 2023-10-19T10:53:23.008 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100841.log 2023-10-19T10:53:23.008 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97177.log.gz 2023-10-19T10:53:23.008 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104696.log 2023-10-19T10:53:23.009 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103946.log.gz 2023-10-19T10:53:23.009 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93740.log: /var/log/ceph/ceph-client.admin.100841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97909.log 2023-10-19T10:53:23.009 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.93740.log.gz 2023-10-19T10:53:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104696.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.100841.log.gz 2023-10-19T10:53:23.010 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99707.log 2023-10-19T10:53:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97909.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104696.log.gz 2023-10-19T10:53:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110675.log 2023-10-19T10:53:23.011 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101804.log 2023-10-19T10:53:23.011 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97909.log.gz 2023-10-19T10:53:23.011 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99707.log.gz 2023-10-19T10:53:23.011 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99413.log 2023-10-19T10:53:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102896.log 2023-10-19T10:53:23.012 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110675.log.gz 2023-10-19T10:53:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101804.log: /var/log/ceph/ceph-client.admin.99413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100379.log 2023-10-19T10:53:23.013 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101804.log.gz 2023-10-19T10:53:23.013 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99413.log.gz 2023-10-19T10:53:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93284.log 2023-10-19T10:53:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-19T10:53:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100379.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102896.log.gz 2023-10-19T10:53:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105148.log 2023-10-19T10:53:23.014 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100379.log.gz 2023-10-19T10:53:23.015 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93284.log.gz 2023-10-19T10:53:23.015 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112068.log 2023-10-19T10:53:23.015 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.105148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96043.log 2023-10-19T10:53:23.015 INFO:teuthology.orchestra.run.smithi050.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105148.log.gz 2023-10-19T10:53:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.112068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96220.log 2023-10-19T10:53:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110306.log 2023-10-19T10:53:23.016 INFO:teuthology.orchestra.run.smithi050.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112068.log.gz 2023-10-19T10:53:23.016 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109147.log 2023-10-19T10:53:23.017 INFO:teuthology.orchestra.run.smithi050.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96043.log.gz 2023-10-19T10:53:23.017 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96220.log.gz 2023-10-19T10:53:23.017 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110306.log: --verbose -- /var/log/ceph/ceph-client.admin.103568.log 2023-10-19T10:53:23.018 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109147.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110306.log.gz 2023-10-19T10:53:23.018 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95917.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109147.log.gz 2023-10-19T10:53:23.018 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.018 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103568.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103568.log.gz 2023-10-19T10:53:23.019 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105232.log 2023-10-19T10:53:23.019 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110222.log 2023-10-19T10:53:23.019 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105232.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94058.log 2023-10-19T10:53:23.020 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95917.log.gz 2023-10-19T10:53:23.020 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110222.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105232.log.gz 2023-10-19T10:53:23.020 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109851.log 2023-10-19T10:53:23.020 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6%/var/log/ceph/ceph-client.admin.94058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101720.log 2023-10-19T10:53:23.021 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.110222.log.gz 2023-10-19T10:53:23.021 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97699.log 2023-10-19T10:53:23.021 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94058.log.gz 2023-10-19T10:53:23.021 INFO:teuthology.orchestra.run.smithi050.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109851.log.gz 2023-10-19T10:53:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101720.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.110096.log 2023-10-19T10:53:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97699.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101720.log.gz 2023-10-19T10:53:23.022 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107155.log 2023-10-19T10:53:23.023 INFO:teuthology.orchestra.run.smithi050.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99455.log 2023-10-19T10:53:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110096.log: -- replaced with /var/log/ceph/ceph-client.admin.97699.log.gz 2023-10-19T10:53:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.107155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93656.log 2023-10-19T10:53:23.023 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110096.log.gz 2023-10-19T10:53:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107155.log.gz 2023-10-19T10:53:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99455.log: --verbose -- /var/log/ceph/ceph-client.admin.110019.log 2023-10-19T10:53:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93656.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99455.log.gz 2023-10-19T10:53:23.025 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111438.log 2023-10-19T10:53:23.025 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110019.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93656.log.gz 2023-10-19T10:53:23.025 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112110.log 2023-10-19T10:53:23.025 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110019.log.gz 2023-10-19T10:53:23.026 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110507.log 2023-10-19T10:53:23.026 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111438.log.gz 2023-10-19T10:53:23.026 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.112110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108885.log 2023-10-19T10:53:23.026 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.110507.log: --verbose -- /var/log/ceph/ceph-client.admin.106618.log 2023-10-19T10:53:23.027 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112110.log.gz 2023-10-19T10:53:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.108885.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110507.log.gz 2023-10-19T10:53:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108885.log.gz 2023-10-19T10:53:23.028 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/ceph-client.admin.111030.log 2023-10-19T10:53:23.028 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99539.log 2023-10-19T10:53:23.028 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106618.log.gz 2023-10-19T10:53:23.028 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108469.log 2023-10-19T10:53:23.029 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111030.log.gz 2023-10-19T10:53:23.029 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95235.log 2023-10-19T10:53:23.029 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.108469.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99539.log.gz 2023-10-19T10:53:23.029 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111354.log 2023-10-19T10:53:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108469.log.gz 2023-10-19T10:53:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.97405.log -- replaced with /var/log/ceph/ceph-client.admin.95235.log.gz 2023-10-19T10:53:23.030 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97867.log 2023-10-19T10:53:23.031 INFO:teuthology.orchestra.run.smithi050.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111354.log.gz 2023-10-19T10:53:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97405.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-19T10:53:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97867.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97405.log.gz 2023-10-19T10:53:23.032 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104114.log 2023-10-19T10:53:23.032 INFO:teuthology.orchestra.run.smithi050.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97867.log.gz 2023-10-19T10:53:23.032 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97951.log 2023-10-19T10:53:23.033 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104486.log 2023-10-19T10:53:23.033 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4%/var/log/ceph/ceph-client.admin.97951.log: -- replaced with /var/log/ceph/ceph-client.admin.104114.log.gz 2023-10-19T10:53:23.033 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103904.log 2023-10-19T10:53:23.033 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.104486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103358.log 2023-10-19T10:53:23.034 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.97951.log.gz 2023-10-19T10:53:23.034 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104486.log.gz 2023-10-19T10:53:23.034 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104798.log 2023-10-19T10:53:23.034 INFO:teuthology.orchestra.run.smithi050.stderr: 94.6%/var/log/ceph/ceph-client.admin.103358.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103904.log.gz 2023-10-19T10:53:23.035 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-19T10:53:23.035 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105106.log 2023-10-19T10:53:23.035 INFO:teuthology.orchestra.run.smithi050.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103358.log.gz 2023-10-19T10:53:23.036 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.104798.log: /var/log/ceph/ceph-client.admin.110946.log 2023-10-19T10:53:23.036 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101104.log 2023-10-19T10:53:23.036 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104798.log.gz/var/log/ceph/ceph-client.admin.105106.log: 2023-10-19T10:53:23.036 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94310.log 2023-10-19T10:53:23.037 INFO:teuthology.orchestra.run.smithi050.stderr: 88.6%/var/log/ceph/ceph-client.admin.110946.log: -- replaced with /var/log/ceph/ceph-client.admin.105106.log.gz 2023-10-19T10:53:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106897.log 2023-10-19T10:53:23.037 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7%/var/log/ceph/ceph-client.admin.94310.log: -- replaced with /var/log/ceph/ceph-client.admin.101104.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-19T10:53:23.037 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.038 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110946.log.gz 2023-10-19T10:53:23.038 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94268.log 2023-10-19T10:53:23.038 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106897.log.gz/var/log/ceph/ceph.log: 2023-10-19T10:53:23.038 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.94310.log.gz 2023-10-19T10:53:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102938.log 2023-10-19T10:53:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.admin.94268.log: --verbose -- /var/log/ceph/ceph-client.admin.101762.log 2023-10-19T10:53:23.039 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.102938.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94268.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110138.log 2023-10-19T10:53:23.040 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.040 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102938.log.gz 2023-10-19T10:53:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.101762.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99791.log 2023-10-19T10:53:23.040 INFO:teuthology.orchestra.run.smithi050.stderr:gzip 87.9% -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-19T10:53:23.041 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101762.log.gz 2023-10-19T10:53:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110138.log: /var/log/ceph/ceph-client.admin.99791.log: 88.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-19T10:53:23.041 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101306.log 2023-10-19T10:53:23.041 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110138.log.gz 2023-10-19T10:53:23.042 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.99791.log.gz 2023-10-19T10:53:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111690.log 2023-10-19T10:53:23.042 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96558.log 2023-10-19T10:53:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/ceph-client.admin.111690.log: -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81723.log 2023-10-19T10:53:23.043 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101306.log.gz 2023-10-19T10:53:23.043 INFO:teuthology.orchestra.run.smithi050.stderr: 87.2%/var/log/ceph/ceph-client.admin.96558.log: -- replaced with /var/log/ceph/ceph-client.admin.111690.log.gz 2023-10-19T10:53:23.043 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-19T10:53:23.044 INFO:teuthology.orchestra.run.smithi050.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96558.log.gz 2023-10-19T10:53:23.044 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph.tmp-client.admin.81723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94226.log 2023-10-19T10:53:23.044 INFO:teuthology.orchestra.run.smithi050.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81723.log.gz 2023-10-19T10:53:23.044 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99497.log 2023-10-19T10:53:23.045 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88107.log 2023-10-19T10:53:23.045 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.94226.log: /var/log/ceph/ceph-client.admin.99497.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94226.log.gz 2023-10-19T10:53:23.045 INFO:teuthology.orchestra.run.smithi050.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99497.log.gz 2023-10-19T10:53:23.046 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108747.log 2023-10-19T10:53:23.046 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94100.log 2023-10-19T10:53:23.046 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.88107.log: /var/log/ceph/ceph-client.admin.108747.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.88107.log.gz 2023-10-19T10:53:23.046 INFO:teuthology.orchestra.run.smithi050.stderr: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.108747.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.93487.log 2023-10-19T10:53:23.047 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109977.log 2023-10-19T10:53:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.94100.log: /var/log/ceph/ceph-client.admin.93487.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94100.log.gz 2023-10-19T10:53:23.047 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95109.log 2023-10-19T10:53:23.048 INFO:teuthology.orchestra.run.smithi050.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93487.log.gz 2023-10-19T10:53:23.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109977.log: gzip -5 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-19T10:53:23.048 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99833.log 2023-10-19T10:53:23.049 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111816.log 2023-10-19T10:53:23.049 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109977.log.gz 2023-10-19T10:53:23.049 INFO:teuthology.orchestra.run.smithi050.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95109.log.gz 2023-10-19T10:53:23.049 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101390.log 2023-10-19T10:53:23.050 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.99833.log: /var/log/ceph/ceph-client.admin.111816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105316.log 2023-10-19T10:53:23.050 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101390.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99833.log.gz 2023-10-19T10:53:23.050 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111816.log.gz 2023-10-19T10:53:23.050 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101390.log.gz 2023-10-19T10:53:23.051 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/ceph-client.admin.101348.log 2023-10-19T10:53:23.051 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105316.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105316.log.gz 2023-10-19T10:53:23.051 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.100883.log 2023-10-19T10:53:23.052 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106758.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101348.log.gz 2023-10-19T10:53:23.052 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.052 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95335.log 2023-10-19T10:53:23.052 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100883.log.gz 2023-10-19T10:53:23.053 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105064.log 87.7% 2023-10-19T10:53:23.053 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106758.log.gz 2023-10-19T10:53:23.053 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96222.log 2023-10-19T10:53:23.054 INFO:teuthology.orchestra.run.smithi050.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95335.log.gz 2023-10-19T10:53:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95151.log 2023-10-19T10:53:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93571.log 2023-10-19T10:53:23.054 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96222.log: /var/log/ceph/ceph-client.admin.95151.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105064.log.gz 2023-10-19T10:53:23.055 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1%gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95151.log.gz 2023-10-19T10:53:23.055 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111942.log 2023-10-19T10:53:23.055 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96222.log.gz 2023-10-19T10:53:23.055 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.93571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101636.log 2023-10-19T10:53:23.056 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93571.log.gz/var/log/ceph/ceph-client.admin.111942.log: 2023-10-19T10:53:23.056 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110801.log 2023-10-19T10:53:23.056 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8%/var/log/ceph/ceph-client.admin.101636.log: -- replaced with /var/log/ceph/ceph-client.admin.111942.log.gz 2023-10-19T10:53:23.057 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109021.log 2023-10-19T10:53:23.057 INFO:teuthology.orchestra.run.smithi050.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101636.log.gz 2023-10-19T10:53:23.057 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94376.log 2023-10-19T10:53:23.057 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.110801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100715.log 2023-10-19T10:53:23.058 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.109021.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110801.log.gz/var/log/ceph/ceph-client.admin.94376.log: 2023-10-19T10:53:23.058 INFO:teuthology.orchestra.run.smithi050.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109021.log.gz 2023-10-19T10:53:23.058 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94376.log.gz 2023-10-19T10:53:23.058 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109893.log 2023-10-19T10:53:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.100715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106378.log 2023-10-19T10:53:23.059 INFO:teuthology.orchestra.run.smithi050.stderr: 88.4%/var/log/ceph/ceph-client.admin.109893.log: -- replaced with /var/log/ceph/ceph-client.admin.100715.log.gz 2023-10-19T10:53:23.059 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101930.log 2023-10-19T10:53:23.060 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109893.log.gz 2023-10-19T10:53:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.106378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105400.log 2023-10-19T10:53:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103400.log 2023-10-19T10:53:23.060 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101930.log: 89.3%/var/log/ceph/ceph-client.admin.105400.log: -- replaced with /var/log/ceph/ceph-client.admin.106378.log.gz 2023-10-19T10:53:23.061 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1%gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105400.log.gz 2023-10-19T10:53:23.061 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.admin.95707.log 2023-10-19T10:53:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103400.log: -- replaced with /var/log/ceph/ceph-client.admin.101930.log.gz 2023-10-19T10:53:23.061 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101062.log 2023-10-19T10:53:23.062 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103400.log.gz 2023-10-19T10:53:23.062 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.95707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111114.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95707.log.gz 2023-10-19T10:53:23.062 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.063 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105190.log 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101062.log.gz 2023-10-19T10:53:23.063 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:23.063 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111114.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-19T10:53:23.063 INFO:teuthology.orchestra.run.smithi050.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111114.log.gz 2023-10-19T10:53:23.064 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.105190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101888.log 2023-10-19T10:53:23.064 INFO:teuthology.orchestra.run.smithi050.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105190.log.gz 2023-10-19T10:53:23.064 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99959.log 2023-10-19T10:53:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97363.log 2023-10-19T10:53:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101888.log: /var/log/ceph/ceph-client.admin.99959.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99959.log.gz 2023-10-19T10:53:23.065 INFO:teuthology.orchestra.run.smithi050.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101888.log.gz 2023-10-19T10:53:23.065 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111648.log 2023-10-19T10:53:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.97363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101432.log 2023-10-19T10:53:23.066 INFO:teuthology.orchestra.run.smithi050.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97363.log.gz 2023-10-19T10:53:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.111648.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111648.log.gz 2023-10-19T10:53:23.066 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96218.log 2023-10-19T10:53:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.101432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103190.log 2023-10-19T10:53:23.067 INFO:teuthology.orchestra.run.smithi050.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101432.log.gz 2023-10-19T10:53:23.067 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.96218.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96218.log.gz 2023-10-19T10:53:23.068 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.103190.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103190.log.gz 2023-10-19T10:53:23.068 INFO:teuthology.orchestra.run.smithi050.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-19T10:53:23.069 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-19T10:53:23.111 INFO:teuthology.orchestra.run.smithi183.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-19T10:53:23.155 INFO:teuthology.orchestra.run.smithi183.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-19T10:53:23.370 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-19T10:53:23.731 INFO:teuthology.orchestra.run.smithi183.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-19T10:53:23.763 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-19T10:53:23.790 INFO:teuthology.orchestra.run.smithi050.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-19T10:53:23.812 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-19T10:53:23.827 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-19T10:53:23.828 INFO:teuthology.orchestra.run.smithi183.stderr: 2023-10-19T10:53:23.829 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m1.087s 2023-10-19T10:53:23.829 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m4.893s 2023-10-19T10:53:23.829 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.193s 2023-10-19T10:53:23.905 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-19T10:53:23.967 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-19T10:53:24.024 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-19T10:53:24.025 INFO:teuthology.orchestra.run.smithi050.stderr: 2023-10-19T10:53:24.026 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m1.310s 2023-10-19T10:53:24.026 INFO:teuthology.orchestra.run.smithi050.stderr:user 0m5.245s 2023-10-19T10:53:24.026 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m0.364s 2023-10-19T10:53:24.027 INFO:tasks.ceph:Archiving logs... 2023-10-19T10:53:24.027 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/remote/smithi050/log 2023-10-19T10:53:24.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-19T10:53:25.026 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/remote/smithi183/log 2023-10-19T10:53:25.028 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-19T10:53:25.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-10-19T10:53:25.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-19T10:53:25.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-19T10:53:25.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-19T10:53:25.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-19T10:53:25.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-19T10:53:25.706 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-19T10:53:25.716 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-19T10:53:25.717 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-19T10:53:25.720 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-19T10:53:25.805 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-19T10:53:25.816 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-19T10:53:25.816 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-19T10:53:25.820 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-19T10:53:25.834 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2023-10-19T10:53:25.836 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2023-10-19T10:53:25.838 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-19T10:53:25.839 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2023-10-19T10:53:25.839 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +2460us[+2463us] +/- 52ms 2023-10-19T10:53:25.839 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -1610us[-1610us] +/- 71ms 2023-10-19T10:53:25.839 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -1963us[-1961us] +/- 77ms 2023-10-19T10:53:25.840 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:53:25.841 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-19T10:53:25.841 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2023-10-19T10:53:25.842 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 +2434us[+2449us] +/- 52ms 2023-10-19T10:53:25.842 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -1621us[-1607us] +/- 71ms 2023-10-19T10:53:25.842 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -1983us[-1968us] +/- 77ms 2023-10-19T10:53:25.843 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-19T10:53:25.844 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-19T10:53:25.858 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-19T10:53:25.860 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-19T10:53:25.872 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-19T10:53:26.001 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-19T10:53:26.102 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-19T10:53:26.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:53:26.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-19T10:53:26.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:53:26.175 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-19T10:53:26.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-19T10:53:26.200 INFO:teuthology.task.internal:Duration was 1493.900730 seconds 2023-10-19T10:53:26.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-19T10:53:26.215 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-19T10:53:26.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-19T10:53:26.218 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-19T10:53:26.262 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-19T10:53:26.267 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-19T10:53:26.507 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-19T10:53:26.508 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2023-10-19T10:53:26.508 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-19T10:53:26.529 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2023-10-19T10:53:26.529 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-19T10:53:26.550 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-19T10:53:26.551 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-19T10:53:26.571 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-19T10:53:26.619 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-19T10:53:26.619 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-19T10:53:26.643 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-19T10:53:26.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-19T10:53:26.772 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-19T10:53:26.772 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-19T10:53:26.803 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-19T10:53:26.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-19T10:53:26.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-19T10:53:26.849 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-19T10:53:26.876 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2023-10-19T10:53:26.896 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2023-10-19T10:53:26.912 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-19T10:53:26.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:53:26.950 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-19T10:53:26.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-19T10:53:26.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-19T10:53:26.977 INFO:teuthology.task.internal:Transferring archived files... 2023-10-19T10:53:26.978 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/remote/smithi050 2023-10-19T10:53:26.978 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-19T10:53:27.031 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271/remote/smithi183 2023-10-19T10:53:27.032 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-19T10:53:27.068 INFO:teuthology.task.internal:Removing archive directory... 2023-10-19T10:53:27.069 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-19T10:53:27.072 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-19T10:53:27.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-19T10:53:27.130 INFO:teuthology.task.internal:Not uploading archives. 2023-10-19T10:53:27.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-19T10:53:27.141 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-19T10:53:27.141 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-19T10:53:27.144 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi050.stdout: 265256 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 19 10:53 /home/ubuntu/cephtest 2023-10-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi183.stdout: 265255 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 19 10:53 /home/ubuntu/cephtest 2023-10-19T10:53:27.174 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-19T10:53:27.184 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-19T10:53:27.208 INFO:teuthology.nuke:Checking targets against current locks 2023-10-19T10:53:27.238 DEBUG:teuthology.nuke:shortname: smithi050 2023-10-19T10:53:27.239 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-19T10:53:27.267 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 10:20:40.335029', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVcW0lHH7vdor8OZ0/odg/BpSOxSYa/C/m1DtWPz9HScDV548DEXi+MR4XATtCyy/F3z0te96Co8q/6rkp4Ffw='} 2023-10-19T10:53:27.269 DEBUG:teuthology.nuke:shortname: smithi183 2023-10-19T10:53:27.269 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-19T10:53:27.299 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-19_10:17:02-fs:functional-wip-63105-distro-default-smithi/7432271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 10:20:40.336202', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFTd5nvqZj0RT84qdMQXVb3bcX1g42IPAWhVYI7TaFUtS+MsLuT/FYYddm4d+bjWgOEDcH+V/K6pd1cofOY7wE='} 2023-10-19T10:53:27.331 INFO:teuthology.orchestra.console.smithi050:Power off 2023-10-19T10:53:27.332 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-19T10:53:27.357 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2023-10-19T10:53:27.358 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:27.375 INFO:teuthology.orchestra.console.smithi183:Power off 2023-10-19T10:53:27.376 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-19T10:53:27.386 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-10-19T10:53:27.399 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-10-19T10:53:27.399 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:27.422 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-10-19T10:53:31.388 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:31.423 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:31.499 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-10-19T10:53:31.534 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-10-19T10:53:35.500 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:35.535 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:35.610 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2023-10-19T10:53:35.647 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-10-19T10:53:35.711 INFO:teuthology.orchestra.console.smithi050:Power off completed 2023-10-19T10:53:39.650 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T10:53:39.761 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-10-19T10:53:39.862 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-10-19T10:53:39.967 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} duration: 1493.9007303714752 failure_reason: 'Test failure: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD)' flavor: default owner: scheduled_dparmar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7b82acfc2b8145d2a15389135e6c8c02 status: fail success: false 2023-10-19T10:53:39.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T10:53:40.044 INFO:teuthology.run:FAIL